Give a model a wall of text, tell it to copy the text back and name the one word that doesn't fit, and vary nothing but the length of the wall. No reasoning, no retrieval, a task a bored human does perfectly. Chroma ran precisely this experiment — they call it "Repeated Words" — across 1,090 variations and 12 input lengths from 25 to 10,000 words. Accuracy fell as the input grew. Not because the task got harder. Because the input got longer.
That result is the cleanest statement I've seen of a problem most of us have felt but not named: a model does not use its context window uniformly. The tokens near the start of a 200k-token prompt are not processed with the same reliability as the tokens near your question. The industry sold million-token windows as if capacity were the same thing as comprehension. It isn't, and there is now a body of research explaining why.
What context rot actually is
Chroma's July 2025 report, Context Rot: How Increasing Input Tokens Impacts LLM Performance, tested 18 frontier models — Claude Opus 4 and the Sonnet line, GPT-4.1 and o3, Gemini 2.5 Pro and Flash, Qwen3 at three sizes — and found the same shape of decay in all of them. Performance degrades as input length increases, and it degrades non-uniformly, in ways the standard Needle-in-a-Haystack (NIAH) benchmark is specifically bad at revealing.
The classic NIAH test hides a sentence in unrelated filler and asks the model to find it. Models ace it, which is why "100% recall at 1M tokens" charts look reassuring. The problem is that NIAH measures lexical retrieval — the needle and the question share words. Real work rarely does. When Chroma varied the conditions to look more like production, the floor dropped out.
The four findings that should change how you build
Semantic similarity between the question and the answer matters more than distance. Chroma measured needle-question cosine similarity across five embedding models (roughly 0.45–0.78 for essay haystacks, 0.52–0.83 for arXiv papers). Low-similarity pairs — where the answer is phrased nothing like the question, which is the normal case — degraded far faster as input grew. Your retrieval hits that hardest exactly when the user asks in their own words.
A single distractor is enough to hurt. Add one topically related but non-answering passage and accuracy drops below the needle-only baseline; add four and it drops further. The failure mode is model-specific: GPT models tend to hallucinate an answer from the distractor, Claude models tend to abstain. Neither is "robust."
Structure can hurt. In the most counterintuitive result, models scored higher on haystacks whose sentences had been randomly shuffled than on the coherent original text. Whatever long-context attention is doing, it is not rewarded by the logical flow we assume helps it.
The RAG-relevant one: LongMemEval. Using 113k-token chat histories, Chroma compared a focused prompt containing only the ~300 relevant tokens against the full history. Every model showed a significant gap. Same information, same question — the only difference was how much irrelevant context surrounded the answer.
"Whether relevant information is present in a model's context is not all that matters; what matters more is how that information is presented."
That line is the whole engineering lesson. More context is not more signal. Past some point it is more noise with a token bill attached.
What to do about it
The fix is not a bigger window; it is a smaller, better-curated payload. Treat the context window as a working set you assemble deliberately, not a bucket you pour logs into.
# Don't: dump the whole history and hope attention finds the needle
prompt = system + "\n".join(all_messages) # ~113k tokens
# Do: retrieve the few spans that answer THIS query, and place them last
spans = retrieve(query, all_messages, k=5) # ~300 tokens
prompt = system + format(spans) + query
Three habits fall out of the research. Retrieve less and tighter — a focused 300-token context beat a 113k-token one, so aggressive filtering is a quality move, not just a cost move. Position the relevant material near the question rather than trusting the model to reach back across the window. And benchmark at your input lengths with your paraphrased queries and distractors — a NIAH pass at 1M tokens says almost nothing about the messy inputs users actually send.
The other rot — and don't confuse them
Context rot is a runtime problem: the weights are fine, the input is the issue. There is a second, deeper failure that shares the naming rhyme and gets muddled with it, and the distinction matters because the fixes are opposite.
A pilot study titled LLMs Can Get "Brain Rot" (arXiv 2510.13928) continually pre-trained four models on low-quality Twitter/X text — operationalized two independent ways, by engagement bait (M1) and by low semantic quality (M2), with token counts matched to a clean control. The junk diet produced a dose-response decline: under M1, ARC-Challenge with chain-of-thought fell from 74.9 to 57.2 and the RULER-CWE long-context score from 84.4 to 52.3 as the junk ratio rose from 0% to 100%, with effect sizes above 0.3 (Hedges' g). The primary reasoning lesion was "thought-skipping" — models truncating or omitting the chain of reasoning altogether. Most sobering: the damage was persistent. Instruction tuning and further clean pre-training only partially recovered it, which points to representational drift rather than a surface habit you can prompt away.
You rarely control pretraining. But you do control fine-tuning sets, synthetic-data loops, and self-training pipelines — and this is direct evidence that feeding a model its own low-quality or engagement-optimized output can leave marks that don't wash out. Curation is not a nice-to-have on the training side any more than it is on the context side.
The takeaway
There are two rots, and one word for "the model got worse" hides them. If quality drops as prompts get longer, that's context rot — reach for retrieval, filtering, and placement, and stop measuring yourself with lexical NIAH. If quality is baked in from what the model learned, that's brain rot — and the only defense is what goes into training. Both point at the same discipline from opposite ends: the model is exactly as good as the tokens you put in front of it, whether that's this request or the last ten million examples. Audit the inputs, not just the outputs.
Sources: Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma Research, 2025) · LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X (arXiv:2510.13928)