Google open-sources Harness²: agents that rewrite their own prompts, tools, and memory at test time

Google Cloud AI Research (with Northwestern University) has open-sourced Harness², a framework that recursively improves an agent's harness — the prompts, tools, memory, skills, and orchestration wrapped around a frozen model — at test time, with no ground-truth verifiers. On three professional benchmarks, a single refinement step improved both GLM-5 and Gemini 3.5 Flash in all four model–executor configurations tested, by up to 11.5 points on WorkBuddy-Bench (paper, repo).

What shipped

Harness² is a framework, not a model: zero parameters are trained. It treats an agent as Agent = Model + Harness, where the harness is the system around the model — system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and a middleware plugin. That is eight editable components, each mounted as a file in the runtime, extending the seven-file split of the earlier Agentic Harness Engineering work by one (paper). The model weights never change; only these files do.

The framework runs two connected levels of recursion (README):

• Task-level recursion refines the harness for the current task through a contrastive Propose–Probe–Compose step. An editor proposes two candidate edits — one "invariant" targeting general failures like planning and tool use, one "domain" targeting the current task's knowledge and procedures. Both candidates run on the same task, and a composer keeps only the edits whose effect is backed by observed behavioral differences between the runs. The contrastive design needs no ground-truth answers: it compares executions against a reference run, not a label.
• Harnessing-level recursion runs after each task. A reflector writes the adopted harness plus an EXPERIENCE.md file of lessons into a persistent improvement library that guides future proposals. Each new task starts from the base harness, but the improvement process carries over.

The recursion budget k scales the search. Parallel mode probes k invariant and k domain candidates and produces k compositions; sequential mode chains k refinement steps (README).

The release is Apache-2.0, runs on Linux with Python 3.12 and uv, and includes a local demo (harness2-demo) that exercises both recursion modes with deterministic editors and real Python executions, no model credentials required. Model-backed benchmark runs go through Vertex AI with OpenCode as the default executor or Codex via --substrate cx (README).

The numbers. The paper evaluates on WorkBuddy-Bench, JobBench, and Harvey's Legal Agent Benchmark (LAB), across Gemini 3.5 Flash and GLM-5 under OpenCode and Codex executors, with results averaged over three runs (paper, MarkTechPost). A single step (k=1) improved every configuration on every benchmark:

Setup	WorkBuddy	JobBench	LAB partial
Gemini 3.5 Flash / OpenCode	72.4 → 77.5	31.5 → 36.0	79.9 → 81.9
Gemini 3.5 Flash / Codex	74.5 → 77.5	33.9 → 35.3	71.6 → 75.6
GLM-5 / OpenCode	64.2 → 75.7	20.6 → 24.2	59.6 → 71.4
GLM-5 / Codex	64.4 → 75.3	23.6 → 24.8	58.1 → 65.9

The largest single-step gain on WorkBuddy-Bench was +11.5 points (GLM-5 under OpenCode), and the largest on LAB was +11.8 percentage points (GLM-5 under OpenCode); the smallest was +1.2 points on JobBench (GLM-5 under Codex). Scaling to parallel recursion at k=5 added up to 3.9 points on WorkBuddy-Bench, 4.2 on JobBench, and 6.7 percentage points on LAB partial pass rate, lifting Gemini under OpenCode to 80.8 on WorkBuddy (paper). The weaker executor gained more: after one step, GLM-5 beat base Gemini 3.5 Flash on WorkBuddy-Bench (MarkTechPost).

Why it matters

The interesting result is not the per-task edits — it is that the meta-level learning carries across tasks. The paper's ablations show the persistent lessons library is worth 3.1 points on WorkBuddy-Bench and 1.8 on JobBench, the scoped invariant/domain editors 2.9 and 3.7, and composition 1.6 and 1.9 (NYU RITS writeup). And the library transfers: a library accumulated on JobBench improves performance on WorkBuddy-Bench (paper). That is the "harnessing" contribution — the improvement process itself generalizes, not just the harness for one task.

For engineers building agents, this is a runnable pattern for self-improvement without retraining or labels. The contrastive Propose–Probe–Compose step is a concrete answer to the open-world problem that most self-improvement methods assume test cases or verifiers that don't exist in production. The same-model-as-executor-and-editor settings the paper reports suggest the loop can run without a frontier editor model, though the default editor is Claude Opus 4.8 (MarkTechPost).

Caveats

• All numbers are vendor-reported. They come from the authors' own paper, averaged over three runs, with no independent re-run found at publication time. MarkTechPost's comparison table against Meta-Harness, ReasoningBank, and Harness Scaling is explicitly "scores as measured in the Harness² paper" — the other methods never reported these benchmarks themselves (MarkTechPost).
• The README itself flags eval-contamination risk. Executor network access is unrestricted and Codex runs with its sandbox disabled; the README warns that public benchmark rubrics may be reachable online and instructs users to configure network restrictions before evaluation (README). Treat the headline gains with that in mind.
• It is not uniformly better than baselines. Meta-Harness stays ahead on LAB for Gemini 3.5 Flash under Codex (NYU RITS).
• Refinement costs compute. Each step runs multiple probe executions of the task, so recursion raises tokens spent before a deliverable is submitted (NYU RITS).
• The default editor is a competitor's model. A Google framework whose harness editor defaults to Anthropic's Claude Opus 4.8 is notable in itself, though the paper also reports same-model editor settings (NYU RITS).
• This is a research release, not a product. The README states it is "not an officially supported Google product" and not eligible for Google's OSS vulnerability rewards program (README). Some practitioners on Hacker News argue the pattern resembles what teams already do — learning from past runs to update prompts and skills — and note the release lacks a single end-to-end runnable example beyond the demo.

Sources

• Harness²: Recursive Agent Harnessing for an Open World (arXiv:2610.14224)
• Full paper text (arXiv HTML)
• GitHub repository: google-research/harness2
• Project page
• MarkTechPost: Google Open-Sources Harness2, a Recursive Agent Harnessing Framework
• NYU RITS: Google Open Sources Harness2 for Self-Improving AI Agents
• Agentic Harness Engineering (AHE), arXiv:2604.25850
• Hacker News discussion