DarwinX: Evolving Agent Harnesses Through Natural Selection
Abstract
DarwinX evolves agent harnesses via population selection with frozen models, improving verified performance across benchmarks without benchmark-specific patches.
An LLM agent's capability depends not only on model weights but on its harness: prompts, tools, skills, and control flow. Self-improvement loops already edit harnesses, yet single-lineage search is path-dependent and local wins often regress other tasks. We introduce DarwinX, which treats self-evolution as selection over a population of harnesses with the model frozen: a preserve-and-extend contract admits only variants that extend coverage without regressing, an archive keeps alternative lineages for recombination, and failure-, teacher-, and self-derived evidence share one edit interface. Fitness comes from each benchmark's own verifier: no gold solutions, no hand-picked winners. Across four benchmarks that progressively separate the evolution signal from the test, one loop adds about 17 points on average: Terminal-Bench 2.1 rises +7.7 to 83.2% on a matched base and to the verified frontier at 84.7% on a stronger one; TerminalWorld's held-out split reaches 68.3%, ahead of every off-the-shelf agent; WebArena-Infinity real-task pass@1 rises from 43.5% to 93.0% audit-clean; and a Terminal-Bench 2.1 harness transfers unchanged to SWE-bench Verified. What evolves is general agent competence, not benchmark-specific patches, so it survives changes of task, verifier, and base model. A frozen model need not be a fixed agent: harness selection turns evaluation compute into durable capability.
Community
We freeze the base model and evolve only the harness — prompts, tools, skills, control flow — by label-free natural selection: variants are scored on measured fitness (avg@k, no gold solutions), survivors are kept, and complementary ones are merged.
Matched-model, harness alone:
- Terminal-Bench 2.1 75.5 → 83.2 avg@5 on GPT-5.5, past Codex at 83.1 (84.7 on GPT-5.6 Sol)
- TerminalWorld 61.0 → 68.3 pass@1 on Opus 4.8, 41 held-out tasks, past Claude Code at 65.9
- WebArena-Infinity 43.5 → 93.0 audit-clean pass@1 on 1,260 real tasks
- SWE-bench Verified 80.8 → 84.2 by zero-shot transfer of the terminal harness
Two results we did not expect. The in-loop proxy saturates (0.505 → 1.000) while held-out pass@1 is 68.3 — a 31.7-point gap — and the variant that best fits the proxy is not the best generalizer, so keeping a population rather than following the single best lineage is what converts an overfit proxy into held-out gain. And we audited every WebArena-Infinity trajectory for validity: the base agent's raw 53.0 falls to 43.5 while ours goes 94.4 → 93.0, so the gap widens under scrutiny instead of narrowing.
Project page with interactive figures, every number regenerated from the run artifacts: https://huggingface.co/spaces/CoderDoge/darwinx
Get this paper in your agent:
hf papers read 2608.07545 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 1
Collections including this paper 0
No Collection including this paper