Context Rot in Long Conversations

Curating the window keeps a fresh task sharp. This lesson is about the opposite situation: a session that has already run long and gone wrong. You curated well, the session ran long, and somewhere in the last forty turns a wrong fact, a stale assumption, or an abandoned approach got written into the context. The model is now anchoring on it, and your instinct is to type the fix. Appending the fix is the natural move, and it does not work, because the wrong fact stays in the window sitting next to your correction.

Context rot is the name for what is happening underneath. A model does not read a long input with even attention, and across 18 leading models the Chroma Context Rot study found that accuracy degrades non-uniformly as the input grows, even on trivially easy tasks. The degradation is not "the relevant fact fell out." A separate EMNLP 2025 result showed drops of 13.9 to 85 percent as input length grew even when the irrelevant tokens were replaced with whitespace and the model was forced to attend only to the relevant span. Length itself is the tax. This kills the comforting assumption that if the information is still in the window, the model will use it.

As the previous lesson noted, the usable window is much smaller than the advertised one, often reliable only to 32K or 64K of a window sold as 200K or a million. What rot adds is the shape of the fall. One study watched Qwen2.5-7B drop rather than slope, its F1 falling from 0.55 to 0.30, a 45 percent collapse, at roughly 40 to 50 percent of its claimed maximum. The same Chroma context-rot study found the decline is non-uniform, with cliffs rather than a smooth ramp: a session that is fine at 50K tokens can turn noticeably worse by 80K. The operator habit is to read that drop as rot, not as the task suddenly getting harder.

The reason the append-a-correction instinct fails sits one level deeper. When you write "actually that was wrong, the deadline is Friday," both the original wrong deadline and your correction are now in the window at once. The model keeps attending to the poisoned tokens because they are still there, still phrased as fact, often still sitting earlier in the context where the model anchored. You are not overwriting the poison. You are sitting it next to its own contradiction and hoping the model picks the right one. The only reliable repair is to make the poisoned tokens leave the window. Rewind to before they entered, restart from a clean summary, or prune them out.

Drew Breunig's taxonomy names four failure modes, and naming the one you are in is the first operator move because the repair follows from the name. Poisoning is a wrong fact or hallucination that got written in and now gets referenced repeatedly. DeepMind's Gemini 2.5 Pokémon agent poisoned its own goals section and stayed fixated on impossible goals for a very long time. The repair is removal, never a correction. Distraction is the window growing so long the model over-focuses on its own accumulated history and stops forming new plans. That same Pokémon agent, past roughly 100K tokens, started repeating past actions instead of synthesizing anything novel. The repair is to compact the history down to decisions and outcomes. Confusion is irrelevant but not wrong content steering the answer. One off-topic sentence appended to a math problem can flip the result. The repair is to prune the noise. Clash is two parts of the context disagreeing. When a task's details arrive spread across many turns instead of one upfront prompt, performance drops 39 percent on average, and o3 fell from 98.1 to 64.1. The mechanism is blunt. When a model takes a wrong turn early in a conversation, it gets lost and does not recover, because the wrong turn stays in the window and biases everything after it. The repair is to start clean with the details consolidated.

Lost-in-the-middle, from the curation lesson, compounds all four. An instruction that was load-bearing forty turns ago now sits in the low-attention middle of a long thread (Liu et al., TACL 2023), technically present and effectively gone. That is not the model disobeying. It is the model not attending to where the instruction ended up.

The hard part is that not every bad answer in a long session is rot, and treating everything as rot wastes the repair. Some perceived degradation is provider-side. The recurring "the model got dumber this week" threads are partly serving-layer load and time-of-day variance, not your context. Teach yourself to treat context discipline as the variable you control without blaming it for everything. The honest tell that it is rot is reproducibility. If a fresh session with the same curated prompt answers correctly and the long one keeps missing, the difference is the accumulated context, and removal is the fix. If the fresh session is also wrong, you have a reasoning or knowledge gap, and rewinding will not touch it.

The other place this breaks is over-aggressive removal. Rewinding or compacting drops tokens, and the subtle, critical detail whose importance only shows up later is exactly the kind of thing a summary quietly loses. Autocompaction makes this worse because it fires when the model is at its least intelligent point, already rotted, unable to predict where your work is heading, so it summarizes a debugging session right before you pivot to the thing the summary dropped. The defense is to never let removal happen automatically mid-task. Compact at a clean boundary, on your terms, and only drop what you can reconstruct. Keep the URL when you drop the page, keep the path when you drop the file. Never discard anything with no path back to it.