Building the same app with 12 JS frameworks costs $2.67 to $4.97 in agent tokens — the bill tracks model familiarity, not framework design
Nolan Lawson's 12-framework agent benchmark: median build cost spans $2.67–$4.97, driven by training-data familiarity and code size, not framework design.
Nolan Lawson had Claude Opus 5.5 build the same Pokémon app five times in each of 12 JavaScript frameworks, then measured median cost, time, tokens, and code size. Median build cost ran from $2.67 (Hono+htmx) to $4.97 (Angular) — a 1.9x spread that tracks how well the model knows the framework, not how "good for agents" the framework claims to be.
What shipped
Lawson's benchmark is a deliberate antidote to vibes-based framework claims. He specced a small but non-trivial PokéAPI app (search, filter, types, moves), then had Opus 5.5 at high effort build it in a sandboxed container using each stack's official MCP or skill where one exists. Each agent got a short prompt — "Implement the app described in SPEC.md using {framework}" — and was tested against a conformance suite built on the accessibility tree rather than the raw DOM, so no framework's HTML structure was mandated.
He took the median of 5 runs per stack — Angular failed to produce a measurable app twice, so it got 3. The repo has the full methodology and prompt.
Median cost per build, in API dollars quoted by Claude Code:
| Stack | Cost | Tokens (M) | Time (min) |
|---|---|---|---|
| Hono+htmx | $2.67 | 4.26 | 8.6 |
| Astro | $2.76 | 4.13 | 8.2 |
| Nuxt | $2.82 | 3.85 | 9.0 |
| Vanilla (Vite) | $2.98 | 5.00 | 10.1 |
| Next.js | $3.14 | 4.99 | 10.9 |
| Qwik | $3.27 | 5.64 | 10.7 |
| TanStack Start | $3.54 | 7.07 | 10.3 |
| SolidStart | $3.79 | 7.13 | 11.2 |
| React Router | $3.81 | 7.58 | 12.5 |
| SvelteKit | $4.28 | 8.53 | 10.8 |
| Gracile+Lit | $4.65 | 9.04 | 12.1 |
| Angular | $4.97 | 9.76 | 17.2 |
The app itself is a content-heavy page, closer to Wikipedia than to Figma or Gmail.
Quality barely separated the stacks. Accessibility scored 100 for all 12, CLS was 0.000 for every stack except Vanilla (0.111), and performance was 97–100. The metric that varied most was shipped JavaScript: Astro sent 0 KiB, Vanilla 7, Gracile+Lit 8, Hono+htmx 18, Qwik 40, Angular 93, React Router 115, TanStack Start 118, Next.js 142.
Why it matters
The headline is not the leaderboard — it's the mechanism behind it. The expensive tail wasn't expensive because those frameworks are verbose. For SvelteKit and Gracile it was unfamiliarity: the agents read the SvelteKit v3 migration guide in every run (too new for Opus 5.5's training data), and for Gracile they either cloned the repo or read its compiled source out of node_modules.
For Angular, Lawson attributes the cost to code size. Either way, the token burn shows up in cost per line of code: Angular produced 2,009 lines for 9.76M tokens, Hono+htmx 1,370 lines for 4.26M — roughly 4,860 vs 3,110 tokens per line, about 1.6x more.
That compounds. Lawson's core argument is that code size is a permanent cost tax: every future agent edit re-reads the codebase. A framework that costs more to build once keeps costing more on every subsequent change. The middle of the table is effectively a tie — but the ends are not, and the ends are where the model's familiarity, not the framework's design, decides the bill.
Two other findings undercut the "better for agents" marketing. First, the official MCPs and skills frameworks ship to help agents were mostly ignored: Nuxt's MCP was never used in any run, Next.js's skill file was read in 1 of 5 runs, TanStack Start's in 2 of 5. Lawson suspects the current crop "may be doing more harm than good." Second, plain Vanilla (Vite, no framework) was only 4th cheapest — having some framework beat having none.
Caveats
This is one author's self-published benchmark, not an independent replication, and the variance is large. All 11 adjacent cost ranges overlap: React Router's cheapest run ($2.82) beat Hono+htmx's most expensive ($3.19). Only the ends — Angular/SvelteKit/Gracile expensive, Hono+htmx/Astro/Nuxt cheap — are defensible; the middle is noise.
It's a single holotype (a content site), a single model, and 5 runs (3 for Angular). Lighthouse and Core Web Vitals are biased against SPAs, and only page loads were measured, not post-load interactions where SPAs win ground back. The familiarity effect is a moving target: a newer model with SvelteKit v3 or Gracile in its training data could flip those results. Treat the dollar figures as illustrative of the mechanism, not a durable ranking.