Cloudflare: how a KV cache leaked 155 responses across customers
A KV cache pool smaller than the context window made Cloudflare's GLM-5.3 return other users' data in 155 responses. How it happened and the SGLang fix.

Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang (Kevin Flansburg and Michelle Chen, Cloudflare, October 26, 2026)
Between August 27 and September 9, 2026, Cloudflare’s hosted GLM-5.3 model sent 155 responses that contained data from other customers’ requests. All of them were answers to prompts longer than 681,280 tokens. This postmortem is the clearest explanation I’ve read of how an inference server can leak one user’s prompt into another user’s answer, and it starts with a number that didn’t match.
Cloudflare advertised a 1,048,576-token context window, but the KV cache pool on its prefill GPUs only had room for 681,280 tokens. The KV cache is the per-token key and value vectors the model attends over, stored in fixed-size GPU memory pages. The post says those pages “are not currently zeroed between requests; the next request that gets a page simply overwrites it.” That’s normal, and it’s fast, and it means the engine has to track exactly which request owns which page.
Here’s the chain, as I understand it from their step-by-step:
- A prompt too big for the pool gets rejected by SGLang’s scheduler and swapped for a one-token placeholder.
- In disaggregated mode, prefill and decode run on separate GPUs. Both admission queues checked whether a request was finished, and a rejected one still looked live.
- The decode node reserved pages for the full prompt, but prefill only wrote one page and reported the transfer complete.
- Decode attended over the other pages anyway, which still held earlier users’ prompts and partial answers.
The client got HTTP 200 and a coherent reply. Cloudflare knew about the size mismatch before launch and assumed the engine would return an error. The sentence I’d pin above any LLM deployment checklist:
This was one critical gap in our testing: we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.
Two things I’m taking from it. If you self-host with disaggregated SGLang, the post recommends v0.5.22 or later and checking KV pool size against context length “on every prefill and decode node, from the engine’s startup log.” And if you only consume an API, this is a reminder that “prompt caching” and “KV cache” are the same machinery. OpenAI’s prompt caching guide says its caches “are not shared across organizations”; Cloudflare’s incident shows why that sentence has to be true at the memory-page level, every time.
Where those cached tokens show up in your bill is the subject of my GPT-6.1 Terra cost post. And for another case this week where the obvious signal said everything was fine, see the Node.js Brotli version check.
Via Hacker News.