Agents refactored 300K lines of C to a perfect Code Health score in three weeks — and wrote their own refactoring playbook

CodeScene published a case study of coding agents refactoring the 300,000-line C codebase of Street Fighter III: 3rd Strike — an open-source decompilation — in three weeks at roughly $4,000 in token costs, moving it from a Code Health score of 5.6 to a perfect 10.0. The more interesting output was not the code but the 22-recipe refactoring playbook the agents discovered and documented themselves.

What shipped

Two engineers did the work — Daniel Webb (CTO at NeoSee) and Dr. Markus Borg — using CodeScene's tooling. The headline numbers, all from the case study: 2,903 commits across 726 files, 252,055 lines modified, and 22 refactoring recipes with 82 supporting notes. The result landed on main via 54 pull requests on a fork, per Webb's reply to a skeptic on LinkedIn. CodeScene founder Adam Tornhill called it "a true superhuman AI performance at scale," after three decades working on large systems.

Two mechanisms carried the work.

A deterministic quality signal. The agents optimized against the CodeHealth MCP Server, a local Model Context Protocol service that exposes CodeScene's CodeHealth analysis to coding assistants — per-file scores, delta reviews, and refactoring business-case estimates, via tools like code_health_review and code_health_refactoring_business_case (docs). That gave them a concrete, deterministic score to maximize and a judge for whether a transformation helped. CodeScene argues the metric is defensible because prior research validated it: the Code Red study of 39 proprietary codebases found low-quality code contains 15x more defects, and resolving them takes on average 124% more time.

An equivalence oracle. Correctness was verified by a replay-trace harness that compared the rollback state hash frame by frame — the game's observable behavior had to stay identical after every change. Webb said it ran as a pre-commit hook, and that some failures were fixed without being observed (InfoQ).

The novel part is the playbook. Instead of applying a fixed catalogue of refactorings, the agents accumulated one, ending with 22 recipes and 82 notes. Some are familiar (Extract Function, Guard Clauses); others are specific to this codebase and, per Tornhill, "structurally different from the code transformations an expert human would consider." Shared Index Range captures repeated loops that differ only in start and end ranges; Action Parameter handles duplicated control structures that differ mainly in which function they invoke; Uniform Step Table turns heterogeneous calls into table-driven dispatch. Failed attempts — transformations that made Code Health worse — were recorded in the playbook too.

Model choice mattered. The team settled on Claude Opus for the bulk of the work, reporting that Claude Code with Opus was "significantly better than Codex with Sol at capturing and documenting the discovered refactoring patterns." Files often plateaued under the smaller Sonnet and Terra models, which "seemed to get stuck at a local optimum" on the remaining code smells.

The uplift was preparation for a study with Lund University: the project produced two functionally equivalent versions of the same system, one at Code Health 5.6 and one at 10.0. Students will next implement features in both using frontier models, comparing cost and quality.

Why it matters

The usual AI-coding story is an agent applying known refactorings faster. This case study inverts it: the agents wrote their own refactoring catalogue, named the recurring transformation shapes they kept hitting, specified preconditions, evaluated whether each worked, and fed the survivors back into the loop. That is a genuinely different capability — the playbook is a reusable artifact, not just a pile of commits, and it is codebase-specific in a way Fowler's catalogue is not.

The architecture is the transferable part: a deterministic quality signal to optimize, plus an equivalence oracle to verify. That combination let the team trust three weeks of largely unattended refactoring. The economics follow — CodeScene frames the $4,000 token cost as about half a monthly developer salary, against an estimated 12–18 months for the same uplift pre-AI (both figures are the vendor's framing, not independent measurements).

The honest tension, which InfoQ's write-up draws out, is that the oracle is the scarce resource. A decompiled fighting game gives you deterministic frame-by-frame replay for free; most legacy systems have no equivalent oracle, which is precisely why refactoring them is risky. Paolo Perrone put the pro case: "most refactor claims i've read rest on a green test suite, which only tells you the tests survived. replaying traces against a fighting game sets a much higher bar." The skeptic case, from Konrad Otrębski: this ran on open-source code, not production code earning money — his proposed bar is offering the same refactoring to a famous open-source project and getting it merged to master.

Caveats

• The two headline ROI figures are projections, not measurements. CodeScene's ~70% reduction in AI-induced defects and ~45% less token waste are extrapolated from its earlier research, not measured in this project. The $4,000 and the three weeks were measured.
• It is a decompiled game fork, not production code. The codebase is the 3SX/3S-decomp project, an AGPL-3.0 fan decompilation of the PS2 game. Webb acknowledged the team picked it partly because they play it and are its users, and that a Gov.UK codebase he had worked on was "too healthy to be useful."
• Diff review is unresolved. Webb raised the question himself — "if you are not familiar with the code and no longer review every line, how big can a diff be?" — and made no assertion.
• Behavioral verification has gaps. Webb said there "might be no definitive answer" on whether the harness caught subtle frame-timing regressions. Non-functional behavior (framerate, memory, input latency) was not assessed in the case study — Marc Bouvier asked about it, and Webb said a performance specialist was being brought in.
• Model vs. harness confound. Asko Nõmm argued that since Claude Code and Codex are tuned to their own models, it is unclear how much of the result measures the model versus the harness. Architecture remains unmeasured — code can look healthy while fundamental problems surface later.
• The recipes are CodeScene's own metric's artifacts. Denis Baltor argued DRY concerns duplication of knowledge and intent, not identical lines, and that recipes defined by loops differing only in ranges may be collapsing the two.

Sources

• Case Study: Refactoring at Scale with Agents — CodeScene
• Agents Refactor 300K Lines in Three Weeks, and Practitioners Ask What It Proves — InfoQ
• CodeHealth MCP Server — CodeScene product page
• CodeScene MCP Server docs
• Code Red: The Business Impact of Code Quality — arXiv
• 3SX repository — GitHub
• CodeHealth as a Prerequisite and Compass for Coding Agents — CodeScene