The long-running agent problem

AI coding agents are now good enough to do real engineering work. They are not yet good at doing a lot of it in a row. A real project, or one large epic, is hundreds of hours of agent work spread over many separate contexts. Over that distance three things fall apart at once: the quality of the agent's reasoning, the faithfulness to what the human actually wanted, and the budget. This page is about that problem, what the field has tried, why it breaks, and a design we built and partly tested to get around it.

Context: we build a real product (a published mobile app) with Claude Code on a flat-rate Claude Max plan, plus OpenAI's Codex as a second reviewer. Everything here is grounded in public research, our own measured runs, and live tests. Estimates are labelled as estimates.

The shape of the problem

An agent does all its thinking inside a context window: the instructions, every file it read, every command output, and its own reasoning, all re-sent to the model on every single step. That window is the agent's working memory, and it behaves badly in three ways that compound:

On top of the memory problem sit two more: verification (an agent's own "done" is not reliable) and coordination (splitting work across many agents creates new failure modes and multiplies cost). Finally there is the human: someone has to define "right" and judge the parts no test can.

What the evidence says

What others have tried, and where each breaks

What is wrong with the usual logic

Measuring a real run

We ran one real epic (adding multiple languages to our app) with one lead chat handing work to helpers, then read every transcript from disk: 44 helpers and about 5,700 model calls over about 21 hours. Cost is in input-token equivalents at the API's price ratios (cache read 0.1x, cache write 1.25x, output 5x).

• Long-lived helpers were the cost, not startup: they ended at a median 316k and up to 816k tokens, after 200-375 steps each.
• Short cache lifetimes were the second cost. Helper caches expired after 5 idle minutes; a long test run or a manager waiting on its builders meant re-writing the whole context. We moved helpers to a 1-hour cache.
• The middle layer was the most expensive agent, because it did hands-on work, held 700-800k, and paid rewrites while waiting.

Replaying the real traces

The obvious fix, "cap each helper's context and restart fresh", seemed sound. We replayed all 26 builders' step-by-step histories under different rules before trusting it:

Fresh restarts lose because getting up to speed is expensive (about 71k at startup, 65k of command and file output, the rest reasoning). Forks win because they start up to speed and read that knowledge from cache at a tenth of the price. That is what pointed us at forking a shared, primed knowledge base.

What we proved about forks

How the design emerged

The solution: knowledge in a seed, work in forks, truth outside

One long-lived scheduler that stays small. A project-manager agent per wave that never writes code. A primed seed session per wave that holds the codebase knowledge and never changes. Short-lived workers that fork the seed, so each starts up to speed for a fraction of the cost. And a fixed quality pipeline, docs, then tests, then code, then adversarial review, where "done" is decided by locked tests and an outside judge, never by the agent that did the work.

Principles

The architecture

flowchart TD
  H["Human owner
approves every Done-when up front
answers parked decisions, walks the result, says push"]
  O["Orchestrator chat
scheduler for the whole epic
never reads or writes code"]
  A["Architect, one per wave
plans, dispatches, verifies,
refreshes the seed. Never codes"]
  GH[("Issue tracker + repo
Done-when, decisions, reports,
canonical docs, locked tests, code")]
  S[("Seed session
the wave's codebase knowledge
primed once, never changes")]
  subgraph F["Forks of the seed, one issue at a time"]
    direction LR
    SW["Spec writer
docs + tests"] --> TR["Test review"] --> CW["Coder
passes locked tests"] --> AR["Adversarial review
+ Codex"]
  end
  H --> O
  O -->|"one per wave"| A
  A -->|"creates, refreshes"| S
  S -. "fork" .-> F
  F -->|"commits, comments"| GH
  A -->|"reads, files issues"| GH

Who does what

How a seed lives

The seed is the heart of the design. It is an ordinary agent session that does one thing: read the wave's ground (the epic, the wave's issues, the architecture docs, the area guide, and every file those issues touch) and then stop. It is never continued. Every agent that needs to understand the codebase starts as a fork of it: a copy of its full conversation, served from the prompt cache, so it begins at the seed's level of understanding for a few hundred new tokens.

flowchart LR
  P["Prime
read epic, issues, docs, files"] --> S1[("Seed v1
commit A")]
  S1 -. fork .-> W1["Issue 1 agents"]
  S1 -. fork .-> W2["Issue 2 agents"]
  S1 -->|"fork + catch up:
git diff A..B, reports"| S2[("Seed v2
commit B")]
  S2 -. fork .-> W3["Issue 3 agents"]
  S2 -. fork .-> W4["Issue 4 agents"]
  S2 -->|"wave ends"| X["Retire.
Next wave primes a clean seed"]

When the architect refreshes the seed

• After a few issues (about three): the code has moved enough that every new worker would re-read the same diffs.
• When a worker reports that the seed misled it, for example a function the seed remembers was renamed.
• When knowledge should be inherited: a research or design report from any model (a Fable brainstorm, a Sonnet investigation) is taught to the seed once, and every later agent inherits it.
• Before splitting into parallel lanes, so every lane starts from the same current understanding.

Why refreshes are not forever

A refresh adds the news on top of the old knowledge; the old view of changed files stays in the seed. After a couple of refreshes, or at the start of a new wave, the architect primes a clean seed from the current code instead.

The quality pipeline: docs, tests, code, adversarial review

Every issue goes through the same fixed order. The architect cannot skip a stage; it decides only how deep the final review goes and whether issues run in sequence or in parallel.

flowchart TD
  DW(["Done-when, approved by the human"]) --> SPEC["1. Spec writer, a seed fork:
updates the canonical docs,
writes tests proving every Done-when line"]
  SPEC --> TREV(["2. Test review: do docs + tests
prove every Done-when line?"])
  TREV -->|"gaps"| SPEC
  TREV -->|"yes"| LOCK["3. Lock: commit spec(#n),
docs + locked tests"]
  LOCK --> CODE["4. Coder, a seed fork:
makes the locked tests pass,
full project gate green"]
  CODE --> AREV(["5. Adversarial review:
seed fork + Codex on spec, tests, diff"])
  AREV -->|"real findings"| CODE
  AREV -->|"pass"| CHECK(["6. Architect verifies: locked tests
unchanged, commit exists, reviews ran"])
  CHECK -->|"ok"| DONE["Issue done, comment on the issue"]
  CHECK -->|"gap, first time"| CODE
  CHECK -->|"second failure"| PARK["Parked for the human"]

Why a separate spec writer

The usual objection to separating test writing from coding is cost and context: the person who thought hardest about the tests has the best understanding for the code. With seeds that objection mostly disappears, because both agents start from the same primed knowledge for a few hundred tokens each. What is left is the benefit: the agent writing the tests cannot shape them around an implementation it already has in mind, and the coder faces a spec it did not write and cannot edit. That is the separation the test-cheating research points at, and it now costs almost nothing. Two locks back it up: a reviewer checks the tests against the Done-when before coding starts, and the architect verifies with git diff that the locked tests did not change.

How deep the final review goes (the architect decides, above a floor)

How the architect orchestrates

• Sequence by default. One issue at a time, each to 100%, following the epic's dependency graph (an issue starts when its blockers are done).
• Parallel when it pays. Issues that touch disjoint files can run in parallel lanes, each in its own worktree with its own seed (the cache is per folder), merged back by the architect.
• Any model off the chain. The architect can start a research or brainstorming agent on whatever model fits, then teach its report to the seed.

How cost stays under control

What stays with the human

• Before: the epic, every "Done when", taste calls, design forks. Front-loading these is what lets a run go unattended.
• During: only parked decisions that are truly irreversible, or truly about taste, money or scope.
• After: walking the result on a real device, and the go to push.

This is the real ceiling of the whole approach: not run length, but the share of work only a human can call "done", and how late they see it.

Running it for real

The exact steps, prompts and commands, using only off-the-shelf tools: Claude Code 2.1.289 on a Claude Max plan (desktop app plus the claude command line), GitHub, and the Codex CLI as the second-family reviewer. No new software.

The graph we are building

An epic is a graph of issues with "blocked by" edges. The orchestrator walks waves; inside a wave the architect walks issues whose blockers are done; inside an issue the pipeline is fixed. Each node is a short loop with a hard check at its end.

flowchart TD
  w1["Wave 1 done"] --> w2["Wave 2: its architect takes over"] --> w3["Wave 3 next"]
  w2 --> s1[("Seed v1")]
  s1 --> na["Issue A"]
  s1 --> nb["Issue B"]
  na --> nc["Issue C, blocked by A"]
  s2[("Seed v2, refreshed after A and B")] --> nc
  nb --> nd["Issue D, new, surfaced in B"]
  nc --> p1["Docs + tests"] --> p2["Test review"] --> p3["Lock"] --> p4["Code"] --> p5["Adversarial review"] --> p6["Architect verifies"]

Before you start

Keeping headless workers out of the memory system

Many long-running setups pair agents with a persistent memory: a hook that captures every finished session and summarizes it into searchable notes. Ours, cc-memory-bridge, captures sessions on Claude Code's SessionEnd hook. That is exactly right for human conversations and exactly wrong for workers: a wave of ten workers plus a seed would leave eleven junk summaries, and over an epic it would drown the real ones.

Headless workers still need the user's settings (the cache setting, permissions, tools), so the hooks cannot simply be turned off. The fix already existed: the bridge's capture step exits silently when CC_MEMORY_BRIDGE_SUBPROCESS=1 is set. It was built so the summarizer's own claude -p calls do not capture themselves; every seed and worker command sets it too. The first trial confirms nothing was captured. Any memory or logging layer used with this design needs the same kind of switch.

Step 0: the human and the agent, together

Write or check the epic, give every issue a "Done when", settle taste calls and design forks now, and mark each "Done when" approved. This is the one place the owner's intent enters; everything later is judged against it.

## Done when (approved)
1. [test] ... a behaviour a test can prove ...
2. [test] ... another one ...
3. [docs] ARCHITECTURE.md describes the new behaviour.
4. [human] (only if it truly needs the owner's eyes) ...

Step 1: the orchestrator prompt

Pasted into a new desktop chat (Opus, high effort). It carries the architect brief, so the architect never depends on the orchestrator's paraphrase.

You're the ORCHESTRATOR for epic <org>/<repo>#<epic>. Read the epic's "Status + start here" block and its latest comment first.

You are a scheduler, not a builder: you never read code, never write it, never run the test suite. You keep the lights on for the whole epic, so keep your own context small - the wave list, which architect is running, and one paragraph per finished wave.

For each wave, in order: start ONE architect (a helper agent, Opus, high effort) and give it the ARCHITECT BRIEF below word for word, with the wave, its issues and the folder filled in. Wait for its WAVE DONE / WAVE BLOCKED report, post one line here, update the epic's latest comment (done, verified, next), then start the next wave's architect.

I'm away: decide like the owner. Write each decision and its reason on the issue. Only stop for something truly irreversible or that truly needs my call - park it as a needs-decision issue and keep going with everything that can move.

Nothing is pushed, merged or deployed until I say so.

[ARCHITECT BRIEF]
[SPEC WRITER BRIEF] [CODER BRIEF] [REVIEWER BRIEF]
[the owner's "How we work" rules, word for word]

Step 2: the architect brief

You are the ARCHITECT for Wave <N> of epic <org>/<repo>#<epic>: issues <list>. Folder: <path>, branch <branch>. You are the wave's project manager. You never write code, and you keep your own context small: reports, issue state, decisions.

1. SEED. In the folder, create the wave's seed once (SEED command). Write the seed id and the commit it read in a comment on the epic.
2. ORDER. Work the issues by their dependencies. Default: one at a time, each to 100%. Run issues in parallel only if they touch disjoint files - each lane in its own worktree, with its own seed.
3. EVERY ISSUE, in this fixed order - never skip or reorder a stage:
   a. SPEC: fork the seed as SPEC WRITER (docs + tests from the Done-when).
   b. TEST REVIEW: a reviewer that sees only the issue, the docs and the tests: do they prove every Done-when line? Gaps go back to (a).
   c. LOCK: the spec writer commits "spec(#n): docs + locked tests". Record the lock commit.
   d. CODE: fork the seed as CODER. It makes the locked tests pass with the full gate green.
   e. REVIEW: at least a Codex pass on the diff. A full adversarial review (a seed-fork reviewer plus Codex, given spec, tests and diff) for anything touching contracts, APIs, schema, security, auth, money or user-facing flows, anything large, and every retry.
   f. VERIFY yourself: the commit exists; git diff --stat <lock commit>..HEAD -- <test files> prints nothing; the reviews ran; the issue has the coder's comment. A gap: one fresh retry. A second failure: label it needs-human and move on.
   Every worker runs in the background (WORKER command) and you wait with the WAIT command.
4. THE EPIC IS ALIVE. Whatever a report surfaces that serves the epic becomes a new issue under it - done in this wave if it belongs here.
5. REFRESH the seed (REFRESH command) after about 3 issues, when a worker says the seed misled it, when a research report should be inherited, or before splitting into parallel lanes. After two refreshes, prime a clean seed instead.
6. WAVE DONE. Reply in at most 5 lines: WAVE DONE or WAVE BLOCKED, issues done and parked, new issues filed, the commits, anything the orchestrator must know.

Step 3: the commands

All in the wave's folder. Same model for the whole fork chain. CC_MEMORY_BRIDGE_SUBPROCESS=1 keeps these sessions out of memory. Workers outlive the 10-minute limit on a single tool command, so they run in the background and the architect polls for their output.

SEED (prime once per wave)

cd "<wave folder>" && mkdir -p runs
export CC_MEMORY_BRIDGE_SUBPROCESS=1
SEED=$(claude -p --model opus --permission-mode bypassPermissions --output-format json \
  "You are the SEED for Wave <N> of epic <org>/<repo>#<epic>. Read, and change nothing: the epic and its latest comment (gh issue view <epic> --comments), the wave's issues <list>, docs/ARCHITECTURE.md, the area guide, and every file those issues name. Learn the conventions and the primitives. Reply only: READY at $(git rev-parse --short HEAD)." \
  < /dev/null | python -c "import sys,json; print(json.load(sys.stdin)['session_id'])")
echo "$SEED"

WORKER (spec writer, coder or seed-fork reviewer: one command, a different brief)

cd "<wave folder>" && CC_MEMORY_BRIDGE_SUBPROCESS=1 claude -p \
  --resume "<seed id>" --fork-session --model opus \
  --permission-mode bypassPermissions --output-format json \
  "$(cat runs/brief-<n>-<role>.md)" < /dev/null > runs/<n>-<role>.json 2> runs/<n>-<role>.err

WAIT

timeout 570 bash -c 'until [ -s runs/<n>-<role>.json ]; do sleep 30; done'; ls -la runs/<n>-<role>.json

REFRESH (fork the seed, let it catch itself up, use it as the new seed)

cd "<wave folder>" && NEW=$(CC_MEMORY_BRIDGE_SUBPROCESS=1 claude -p \
  --resume "<seed id>" --fork-session --model opus --permission-mode bypassPermissions --output-format json \
  "You are being refreshed into the new seed. Run git diff <seed commit>..HEAD --stat, read the changed files that matter, and read these reports: <issue numbers or files>. Change nothing. Reply only: READY at $(git rev-parse --short HEAD)." \
  < /dev/null | python -c "import sys,json; print(json.load(sys.stdin)['session_id'])")
echo "$NEW"   # from now on, fork this id instead of the old one

Refreshing costs only the news: the fork reads the old seed from cache and pays for the diff and the reports it reads. The old seed stays untouched, so a refresh can never corrupt what earlier workers relied on.

Step 4: the role briefs

SPEC WRITER

You are the SPEC WRITER for issue <org>/<repo>#<n>. Your view of the code is from commit <seed commit> - first run git diff <seed commit>..HEAD --stat and read what changed that touches this issue. Read the issue and its comments: its Done-when is the contract.

1. Docs first: update the canonical docs so they describe the behaviour this issue must have.
2. Tests: write tests that prove every Done-when line against those docs. They must fail now (no implementation yet).
3. Do not write implementation code.
4. Reply in at most 200 words: the docs changed, the tests added, which Done-when line each test proves, anything unclear you decided and why.

CODER

You are the CODER for issue <org>/<repo>#<n>. Your view of the code is from commit <seed commit> - first run git diff <seed commit>..HEAD --stat. The spec is locked at commit <lock commit>: the docs and tests there define done.

1. Make the locked tests pass and the full project gate green. You may ADD tests; you may not weaken or edit a locked test or the spec docs. If one is truly wrong, stop and say why in your report.
2. Clean, DRY, reuse and grow existing primitives - never a one-off. Use helpers as you see fit (any model) for research or grunt reading.
3. Commit. Post a comment on the issue: what changed, the commits, the test numbers.
4. Reply in at most 300 words: DONE / BLOCKED / FAILED, the final commit, the issue comment link, new issues worth filing, anything the seed got wrong.

REVIEWER (seed fork or fresh; Codex gets the same request)

You are an adversarial reviewer for issue <org>/<repo>#<n>. You change nothing and start no agents. You are given the Done-when, the locked spec (docs + tests at <lock commit>) and git diff <lock commit>..HEAD.
Judge only against the Done-when and the spec: PASS or FAIL per line with evidence (file:line, test name or command output). Then list any other real problem with its severity: correctness, security, duplication of an existing primitive, docs out of step with code. Do not soften a FAIL.

What the human sees

• In the orchestrator chat: one line per finished wave, then a final report: issues done, issues parked, new issues filed, the branch, nothing pushed.
• In the issue tracker: the epic's latest comment (done, verified, next), each issue's spec and coder comments, a short needs-decision list.
• Their part: answer the parked decisions, walk the result, say "push".

The real limits

The trial

1. When: after the current epic run finishes (nothing about a running epic changes mid-run).
2. What: one wave of the next epic, every Done-when approved up front.
3. First checks: background workers and the wait work from inside an architect; forks read the seed from cache; nothing lands in the memory system.
4. Measure: quota per issue against our baseline, the orchestrator's and architect's context size, what the test review and the adversarial review caught, how many decisions truly needed the human.
5. If it holds: it becomes the default way large epics run.

For reviewers: attack this

We want this design taken apart before we rely on it. Below: what is measured versus assumed, the weak spots we already see, and the specific questions we most want challenged. Where you think we are wrong, say why, and point at evidence if you have it.

Measured, tested, or assumed

Weak spots we already see

Questions we most want challenged

1. Static seed versus a living expert. Is a primed, never-changing seed plus refreshes really better than one long-lived expert agent with good compaction? What would make it worse?
2. Is the architect a middle layer in disguise? The evidence says middle layers cost the most and help the least. We kept one per wave because without it the scheduler drowns in detail. Is there a better split?
3. Spec writer separate from coder: worth the extra step, or does the coder's own test-first discipline plus locking get the same result?
4. Reviewer independence: is a fork of the same seed independent enough to be an adversarial reviewer, or must the judge always start fresh?
5. Refresh cadence: what signal should trigger a refresh, and when does a refreshed seed become worse than a clean one?
6. Parallelism: we default to one issue at a time. Where would you parallelize, and how would you handle merges without a costly integrator agent?
7. Measuring "green but wrong": what metric would show that the pipeline catches work that passes tests but misses intent?
8. Failure modes we have not imagined: what breaks first at 50 issues? At 200?

Sources

• Chroma, "Context Rot" (2025). Du et al., EMNLP Findings 2025, arXiv 2510.05381. Laban et al., arXiv 2505.06120.
• Factory, "Evaluating context compression" (Dec 2025). Codex GitHub issues on post-compaction loss.
• METR time-horizon data (2026); METR, "Towards reconciling slowdown with time horizons" (Aug 2025).
• ImpossibleBench, arXiv 2510.20270. TDFlow, arXiv 2510.23761.
• MAST, "Why Do Multi-Agent LLM Systems Fail?", arXiv 2503.13657. Google/MIT on agent scaling, arXiv 2512.08296.
• Anthropic engineering: multi-agent research system; effective harnesses for long-running agents; harness design for long-running apps; building a C compiler with agents. Cursor, "Scaling long-running autonomous coding" (Jan 2026). Cognition, "Don't build multi-agents" and "Multi-agents: what's actually working" (2026). OpenAI, long-horizon Codex runs. Geoffrey Huntley on Ralph; RepoMirror; DoltHub on Gas Town.
• Claude Code documentation: sub-agents and forks, prompt caching, agent teams, desktop scheduled tasks.
• Our own figures: transcripts of one epic run, 12 Oct 2026, analysed with our scripts.