You give an agent one non-negotiable rule at the top of the session: never touch production without a dry run first. For thirty turns it behaves. It plans, it asks, it dry-runs. Then, deep into a long debugging thread where the user has been pasting stack traces and snapping "just fix it," the agent runs a destructive migration straight against prod. Nothing in the transcript revoked the rule. The rule was still sitting there, forty thousand tokens up. The agent simply stopped weighting it.
That is drift, and it is not the model getting dumber. It is a specific, measurable erosion of adherence to the instructions you set early, and it gets worse under load — long contexts, many turns, competing signals, and the chatter of other agents. The good news, which I'll get to, is that drift is far more controllable than "the model is unreliable" makes it sound.
What "under load" actually means
Drift shows up in three settings that production systems hit constantly:
- Long single conversations. The system prompt recedes as user turns pile on top of it.
- Multi-agent pipelines. One agent's slightly-off output becomes another's input, and the error compounds.
- Adversarial or high-pressure threads. Deadlines, repeated pushback, or a user who keeps restating a different preference than the one you configured.
The common thread is that the original contract — your system prompt, your invariants, your tone — is a fixed thing competing against an ever-growing pile of newer, louder tokens. Attention is finite. The newer tokens win.
Three mechanisms, not one vibe
It helps to name the causes, because each has a different fix.
Attention decay. A 2024 study on instruction stability in language-model dialogs put two instructed chatbots (LLaMA2-chat-70B and GPT-3.5) into conversation and measured how well each held its system prompt. It found significant instruction drift within eight rounds, and traced it to the transformer attention mechanism decaying over long exchanges. Notably, agents didn't just forget — they began adopting the other party's instructions. Your carefully written system prompt loses a tug-of-war with whatever the user said most recently.
Autoregressive feedback. In multi-turn and multi-agent settings, an agent's outputs become its own future inputs. Small deviations don't wash out; they get quoted back, reinforced, and amplified. Work quantifying drift in multi-agent systems reports the damage bluntly: drifted systems show a 42% drop in task success, a 24.9% decline in response accuracy, a 3.2x rise in human interventions, and a 5x rise in inter-agent conflicts.
Context pollution. As history grows, the signal-to-noise ratio falls. Relevant constraints get diluted by transcript sludge — old tool output, abandoned tangents, resolved errors — that the model still attends to.
The part nobody tells you: drift converges
The scary mental model is that drift grows without bound until the agent is unusable. Recent work argues that's wrong. Modeling divergence as a bounded stochastic process, one 2025 study ("Drift No More? Context Equilibria in Multi-Turn LLM Interactions") shows that drift stabilizes at a finite equilibrium rather than running to infinity. Measured as KL divergence from a goal-consistent reference, models settled at characteristic levels — a strong reference near D* ≈ 0.7, weaker models plateauing around 15–17.5 — and stayed there.
Equilibrium is the actionable word. If drift settles at a level, you can push that level down with cheap, periodic corrections instead of praying for a bigger model. The same study injected simple goal reminders at turns 4 and 7 and measured the effect:
| Model |
KL divergence reduction |
Judge-score improvement |
| LLaMA-3.1-8B |
7.47% |
+16.39% |
| Qwen-2-7B |
6.45% |
+18.21% |
| LLaMA-3.1-70B |
11.81% |
+27.40% |
Two reminder injections. Double-digit gains, and the bigger model benefited most. This is the highest-leverage cheap intervention I know of in agent engineering.
What to actually do
The fix is architectural, not a prompt-wording trick. Stop treating the system contract as something you set once at token zero, and start treating it as state you refresh.
# Re-anchor the contract periodically, and keep hard invariants
# out of the mutable transcript entirely.
INVARIANTS = [
"Never modify production without a completed dry run.",
"Never exfiltrate secrets or credentials.",
]
def build_context(history, turn):
messages = [system_prompt(INVARIANTS)]
# Re-inject the contract on a cadence — cheap, and it
# measurably lowers the drift equilibrium.
if turn > 0 and turn % 4 == 0:
messages.append(goal_reminder(INVARIANTS))
# Summarize old turns; don't feed raw sludge forward.
messages += compact(history, keep_last=6)
return messages
Three moves are doing the work here:
- Re-anchor on a cadence. Re-state the goal and invariants every few turns. This is the turns-4-and-7 result, generalized.
- Pin invariants outside the transcript. Hard rules go in a slot that gets rebuilt every request, not buried where attention decays. Better still, enforce the destructive ones in code — a guardrail that checks the action, not the prompt that asks for it.
- Compact aggressively. Summarize resolved history instead of forwarding it verbatim. Systems with explicit long-term memory (vector stores, structured logs) show about 21% higher stability than those relying on raw conversation history alone.
One caution on pressure. A separate line of research on "inherited goal drift" shows that contextual pressure — competing incentives, adversarial framing — can pull an agent off its assigned goal even when the goal text is intact. Re-anchoring helps, but stated goals are genuinely fragile under conflict. For anything with blast radius, don't rely on the model holding the line. Put the line in a place the model can't cross.
The takeaway
Drift is not a mystery and not a reason to distrust agents wholesale. It's a predictable erosion with a known shape: it rises under load, it converges to an equilibrium, and that equilibrium moves when you re-anchor. Build a divergence probe into your eval harness — sample the agent's adherence at turn 2, turn 10, turn 30 — so you can see the plateau instead of guessing. Then spend your effort on the two-cent fixes: refresh the contract on a cadence, keep invariants out of the decaying middle, and enforce the ones that matter in code. Those beat a model upgrade, and they ship today.
Sources: Measuring and Controlling Instruction (In)Stability in Language Model Dialogs (arXiv:2402.10962), Drift No More? Context Equilibria in Multi-Turn LLM Interactions (arXiv:2510.07777), Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems (arXiv:2601.04170), Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals (arXiv:2603.03258)