In brief: Context rot is the silent drop in an AI agent's accuracy as its context grows. Nothing errors; the answer is simply more likely to be wrong. This guide covers the causes, how to spot it, Context Hygiene (seven controls that reduce it) and how to test your own agents.
Who this is for: heads of AI platforms, data and AI engineering leads, and CIOs moving agents from pilots onto enterprise data.
Context rot is the drop in an AI model's accuracy and reliability as the amount of text in its context window grows. The model has not changed and the task has not changed, but the more it is given to read, the more likely it is to miss the fact that matters, follow an out-of-date instruction, or answer from the wrong document.
The term became common after a July 2025 research report from Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance, which tested 18 leading models and found that performance fell as input length rose, even on simple tasks and well before the context window was full.
Why a bigger context window does not solve it
A large context window says how much a model can accept. It says nothing about how well the model uses what it accepts. Teams often read "one million tokens" as permission to send everything: the whole policy library, the full ticket history, every tool result. The model accepts it all and then performs worse than it did with a short, well-chosen input.
Context rot vs context overflow
| Context overflow | Context rot | |
|---|---|---|
| What happens | The input is larger than the model's limit | The input fits, but the model uses it badly |
| How you find out | An error, or text is cut off | Nothing fails; the answer is confidently wrong |
| When it starts | At the limit | Well before the limit |
| Fix | Send less, or use a larger window | Send better: select, order and refresh the context |
Overflow is easy to catch because it is loud. Context rot is the more expensive problem because it is silent.
Lost in the middle
Models do not read every part of a long input equally. Research on long-context use, including the paper by Liu and colleagues, Lost in the Middle: How Language Models Use Long Contexts, found that models tend to use information at the start and end of the input more reliably than information in the middle. A critical rule placed on page 40 of 80 is the one most likely to be missed. This is why the advertised size of a context window overstates how much a model can use well.
Context rot vs context bloat, poisoning and collapse
These four terms are often mixed up. They are different problems with different fixes.
| Term | What it means | Example |
|---|---|---|
| Context rot | Accuracy falls as the input grows, even when the right facts are present | An agent applies last year's limit because both versions were in its context |
| Context bloat | More text in the context than the task needs; a main cause of context rot | The full ticket history is sent to answer one billing question |
| Context poisoning | A wrong fact, often an earlier mistake, enters the context and is reused | An agent misreads an account number in step 2 and relies on it for every later step |
| Context collapse | Details are lost when an agent repeatedly summarises or rewrites its own history | After several summaries, the exception agreed for one customer has disappeared |
What causes context rot
- Distractors. Text that is close to the right answer but wrong, such as last year's policy next to this year's, pulls the model toward the wrong one.
- Buried facts. A single relevant line among thousands of irrelevant ones is easier to miss, especially in the middle of a long input.
- Conflicts. Two sources that disagree force the model to pick, and it does not reliably pick the authoritative one.
- Stale history. In a long agent run, early instructions, abandoned plans and old tool results stay in the context and keep influencing later steps.
- Noise from tools. Raw API responses and logs fill the window with material the task never needed.
How to recognise it in an AI agent
- The agent works well on short tasks and fails on long ones of the same kind.
- It repeats a step it has already done, or returns to a plan it had dropped.
- It quotes a rule that has been replaced.
- Its answers change when unrelated documents are added to the prompt.
- Accuracy falls as a session goes on, then recovers when the session is restarted.
Symptoms and what they point to
| What you see | Likely cause | What to test |
|---|---|---|
| Right answer on short input, wrong on long | Length itself | Run the same task at three input sizes |
| Agent applies an old rule | Superseded versions still in context | Remove old versions and re-run |
| Agent ignores a clear instruction | Instruction buried mid-context | Move it to the start or end |
| Answers change when unrelated files are added | Distractors | Tighten retrieval and compare |
| Quality drops late in a session | Accumulated history and tool output | Summarise or restart and compare |
How to reduce context rot: Context Hygiene
We group the fixes into seven controls and call them Context Hygiene. Each one can be tested on its own. Anthropic's guidance on effective context engineering for AI agents makes the same point: treat context as a limited resource and give the model the smallest set of high-signal information that does the job.
- Select, do not dump. Retrieve only what this decision needs. Fewer, more relevant passages beat a full archive.
- Put authority in the context. Mark which source wins when two disagree: current policy over old, system of record over a copy.
- Remove what has expired. Drop superseded versions, finished sub-tasks and old tool output instead of carrying them forward.
- Summarise long runs. Replace a long history with a short, checked summary of what was decided and why.
- Split the work. Give sub-tasks their own clean context and pass back only the result.
- Place by position. Put the instructions and the facts that matter most at the start or end of the context, not in the middle.
- Measure by length. Test the same task at short, medium and long input sizes. If accuracy falls with length, the problem is the context, not the model.
Find where context rot hits your agents. In a one-day context rot review we take one agent workflow, run it at short, medium and long input sizes, find where accuracy drops, and show which sources, versions and tool outputs to cut.
Context rot: then, now and next
- 2023: the Lost in the Middle study shows models use the start and end of long inputs more reliably than the middle.
- 2024: Google announces Gemini 1.5 with a context window of up to one million tokens, and "send everything" becomes tempting.
- July 2025: Chroma tests 18 models and gives the problem its name.
- September 2025: Anthropic describes context engineering as the discipline of choosing what goes into the context.
- 2026: agents move from pilots onto ERP, CRM and payment systems. A wrong rule in context now means a wrong action, not just a wrong answer.
- 2027 onwards: under the EU AI Act, high-risk systems must keep automatic records (Article 12), with Annex III obligations applying from 2 December 2027. Being able to show what information an agent worked from becomes a compliance question as well as a quality one.
The cost of context rot
Long context is paid for twice. Every extra token is billed and slows the response, and the lower accuracy it causes creates rework: retries, human review and corrections downstream. Sending less, better-chosen context usually improves quality and cost together.
Context rot in the enterprise
In a demo, an agent reads ten documents. In production it reads from an ERP, a CRM, a ticketing system, a policy library and a data warehouse, each with its own versions and its own idea of what is true. That is where context rot does real damage: an agent that approves a payment under a limit that was lowered last month has not hallucinated. It read the wrong rule because both rules were in front of it.
This is the problem context engineering exists to solve, and it is the design idea behind ElixirData's Context OS: assemble the context for each decision at the moment of the decision, from governed sources, with the current policy and the right authority attached and checked before the agent acts, instead of handing the agent everything and hoping it chooses well. Recording past decisions in a context graph lets an agent pull a few linked facts in place of pages of raw documents. Keeping a decision trace for each action also shows, afterwards, which context the agent actually used, so you can tell whether context rot caused a wrong decision.
Governed context assembly, for each decision:
ERP, CRM, tickets, policy library
only what this decision needs
current policy and system of record win
drop old versions and stale tool output
after a policy check
records the context it used
Frequently asked questions
Is context rot the same as hallucination?
No. A hallucination is an answer with no basis in the input. With context rot the right information is usually present; the model fails to use it because of everything else around it.
Does retrieval (RAG) fix context rot?
It helps when retrieval is precise. Retrieval that returns many loosely related passages adds distractors and can make it worse.
Do newer models fix it?
Newer models tolerate longer inputs better, but the Chroma report found the decline across all the models it tested. Treat context length as something to manage, whatever the model.
How long is too long?
There is no single number. It depends on the model, the task and how much of the input is relevant. The only reliable answer comes from testing your own task at different lengths.
Is context rot still a thing with newer models?
Yes. Larger context windows raise the limit on how much a model can accept, but they do not make it use long inputs equally well. Test your own model on your own task at short, medium and long input sizes before trusting a large window.
What is context bloat?
Context bloat is putting more into the context than the task needs: full histories, whole documents, raw tool output. It is one of the main causes of context rot.
What is an example of context collapse?
An agent on a long case summarises its history every few steps to save space. After several rounds, the one-off exception agreed for a customer is no longer in the summary, and the agent stops applying it. Keep key decisions and exceptions as structured records, not only in free-text summaries.
Do we have to move our data to fix context rot?
No. The fixes are about what goes into the context for each decision, not where the data lives. Context can be selected and assembled inside your own environment.
Who should own it?
The team that runs the agent platform, working with the owner of each data source. Business owners decide which source wins when two disagree.
Next step: Book a one-day context rot review → or see Context OS in a demo.
Related on ElixirData: Context Rot
Sources
- Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance, July 2025.
- Anthropic, Effective context engineering for AI agents, 29 September 2025.
- Google, Our next-generation model: Gemini 1.5, 15 February 2024.
- Liu et al., Lost in the Middle: How Language Models Use Long Contexts, 2023.
