Context and compaction

In short: every model has a limit on how much it can read at once. A long conversation, or an agent that's been working for an hour, eventually hits it — and then it stops working. Compaction summarises the older part of the history so the run can carry on. agentFast does this for you, tells you when it happened, and keeps a copy of what was dropped.

The problem

An agent's context is everything it can see right now: the instructions, the conversation so far, every tool call it made and every result that came back. That grows with every step.

Models have a hard ceiling on that. Cross it and the provider rejects the request outright — not a degraded answer, an error. The two shapes of work that hit it are exactly the two shapes people build:

• Long conversations. A support thread that runs for forty turns.
• Long autonomous runs. A research or coding agent working for an hour, accumulating tool results the whole time.

There's a cost dimension too. You pay for input tokens on every single call, so a history you never trimmed is a history you re-pay for on every turn. Compaction is a cost control before it's a capacity control.

What agentFast does

Before each model call, the runtime estimates how big the history is. Under the threshold, nothing happens. Over it, the history is compacted and the run carries on with a smaller one.

Two modes, because the two shapes of work fail differently:

The distinction matters. In a support conversation, "the customer just said they'd already tried restarting" is exactly the kind of detail a summary flattens — so tiered keeps recent turns untouched. In an hours-long coding run, the deliberation is noise and the conclusions are the asset — so reset throws away the reasoning and keeps what was established.

Configuration

compaction:
  enabled: auto           # auto | on | off
  mode: reset             # reset | tiered
  threshold_tokens: 100000
  keep_recent: 6          # tiered only — turns kept verbatim
  retain_history: true    # keep the dropped messages in the episodic log

auto means: compact when a model is available to write the summary, and don't when one isn't. Compaction needs a model call, so a keyless quickstart legitimately runs without it. Set on to be told loudly when the key is missing rather than quietly running uncompacted.

The summariser uses the harness_llm settings — the same model agentFast uses for its own calls, separate from the model your agent reasons with. Set compaction.model to override just this one, which is usually worth it: summarising is frequent and mechanical, so a cheap fast model is the right trade.

You can see it happen

Compaction is agentFast spending your money on your behalf, so it shows up in the trace as its own step type rather than being folded into your agent's cost:

If the run is being watched over SSE, the same event goes to the stream — so a UI shows "compacted context — 104,218 → 1,940 tokens" instead of going silent for several seconds while the framework summarises.

What was dropped is kept

Compaction rewrites the message list, and that list is what gets checkpointed. So a compaction followed by a crash means the run resumes from the compacted history — correct, but it makes the checkpoint lossy. Without a record, "why did the agent forget that?" would be unanswerable after the fact.

So the pre-compaction history is written to the episodic log before it's replaced. Resume behaviour is unchanged; only the audit gains. The record is size-capped and says explicitly when it was truncated, so you can tell "this is all of it" from "this is what we kept".

Set retain_history: false where message contents are too sensitive to sit in a log at rest. You still get the step, the token counts and the timing — you lose only the message text.

Honest limits

Token counts here are estimates (characters ÷ 4). Deciding whether to compact is a budget question, not a billing one, and an estimate is enough for it. The authoritative numbers — the ones in your step tree and your invoice — come from the provider and are exact.

Related