Originally published on Loop & Retry — field notes on building LLM agents that survive production.
Every long-running agent eventually hits the wall: the context window fills, and something has to give. The near-universal fix is compaction — summarize the older turns into a shorter recap, drop the raw transcript, and keep going. It works, right up until the run where the agent cheerfully violates a constraint it was given on turn 3, because turn 3 didn't make it into the summary. Nothing errored. The agent just forgot, and forgot in a way that looks exactly like a reasoning failure instead of the data-loss bug it actually is.
This post is about treating compaction as what it is: a lossy compression step in the middle of your control flow. I've argued before that the context window is a cache, not a memory — that you should run it with a budget and an eviction policy. Compaction is that eviction policy, executed by a model that doesn't know which facts are load-bearing. That's the problem.
Why "summarize the old turns" loses the wrong thing
Compaction usually works on recency. The last few turns stay verbatim; everything older gets squeezed into a paragraph or two of summary. This is a reasonable default for the shape of a conversation — recent context is usually most relevant to the next step — and it is exactly wrong for a specific, common, and costly case.
The facts that matter longest are often stated earliest. The user's hard constraint ("never touch the production database", "the budget is $500, hard cap", "the customer is in the EU so GDPR applies") arrives at the start of the task and stays relevant until the end. A recency-based summarizer sees that fact age out of the verbatim window, tries to compress it alongside fifty other turns of tool calls and chit-chat, and — because a summary's whole job is to drop detail — renders it as "the user described some requirements" or drops it entirely. The load-bearing constraint gets the same treatment as the small talk.
The failure is delayed and disguised. The agent runs fine for eighty turns. Then it reaches the step where the dropped constraint would have applied, doesn't have it, and does the reasonable-looking wrong thing. You debug it as a reasoning error or a prompt problem. It's neither. It's a fact that was in context, got compressed out, and never came back.
Continue reading: the rest of this post — including the sections that follow — is on Loop & Retry: read the full article
Top comments (0)