AI · 05
Context rot
Why a long chat or a huge prompt makes AI less reliable — even when the answer is right there in front of it.
The one idea
More context is not free. As the context window fills up, a model gets worse at finding, weighing and using what's in it. The right fact can be sitting in the prompt and still be missed.
A bigger window means it fits more — not that it uses it all well.
What it looks like
- Missed details — something buried in the middle of a long prompt gets ignored.
- Contradictions — it goes against an instruction or fact you gave earlier.
- Drift — over a long conversation it wanders from the original task.
- Distraction — similar-looking but irrelevant content pulls the answer off course.
Likely causes
Researchers still debate the exact mix; these are the usual explanations.
How to keep it sharp
- Start fresh for a new task instead of letting one thread sprawl.
- Summarise, don't carry — compact earlier history into the key points and continue from that.
- Put what matters at the start or end, and restate key instructions when you need them.
- Retrieve, don't stuff — give it the relevant passages (RAG), not whole documents.
- Trim the noise — cut logs and pasted output down to what the task needs.
If you work with an AI coding assistant
A model can't reliably feel rot from the inside — it doesn't know what it has stopped paying attention to. So give it outside signals, and ask it to watch for visible symptoms in its own work rather than trusting it to sense the decline.
Symptoms to watch for:
- Forgetting earlier decisions or re-asking questions you already answered.
- Contradicting its own plan, or repeating a mistake it already fixed.
- Re-reading the same files or editing the wrong one.
The term was popularised by Chroma's 2025 research, which found that models with very large context windows still get worse at simple tasks as the input grows.