openTPU: AI agents wrote the RTL, ISA, and compiler for the chip that runs their own inference — bit-exact across ten models on a Kintex-7

openTPU — an open-source inference accelerator whose SystemVerilog, instruction set, compiler, and host stack were all written by AI agents — now runs ten models with their real weights on a Xilinx Kintex-7 PCIe card, producing the same tokens as its Python reference simulator bit for bit. It is one Apache-2.0 monorepo, and everything except the card runs on a laptop.

What shipped

openTPU (by developer FeSens) is an inference accelerator built around a deliberately simple machine: a sequencer issues one instruction per cycle to DMA, an int8 systolic matrix unit, an fp32 vector unit, and a quantizer. The ISA is eight 32-bit words per instruction; there is no cache and no hidden scheduling, so every data movement is an instruction and a trace shows exactly where cycles went. The monorepo holds the SystemVerilog RTL, the ISA, a bit-exact Python simulator, a kernel language and compiler (@ol.jit), host tools (otpu-chat, otpu-serve, otpu-smi, otpu-selftest), and a browser-based profiler, Lens, that replays runs from the RTL, the simulator, or the card with a roofline and per-instruction timelines (README).

The target is an Inspur YPCB-00338 PCIe card: a Xilinx Kintex-7 xc7k480t with two DDR3-1066 channels (4 GiB, 17.1 GB/s peak). One bitstream at 133.33 MHz serves every model. The memory controllers are LiteDRAM, calibrated at boot by a small CPU inside the memory core — both channels in 12 seconds, no host involvement (README).

The project's stated questions are "how far can AI agents go at hardware design" and "can they build the chip that runs their own inference" (README). On Hacker News, where the repo drew 348 points and 405 comments, the author called it the "first opensource TPU build by a recursive improvement loop," and said a second generation "could run the inference of the models that are being used to improve it" (comment). He frames the FPGAs as architecture proof "before committing 100's of millions into a custom ASIC with TSMC" (comment). The method extends his earlier auto-arch-tournament project, previously applied to AI-designed RISC-V cores (AICoder).

Verification

When an AI writes the RTL, the question is whether silicon does what the spec says. openTPU's answer is an executable spec and a chain of bit-exact equivalence: a kernel compiles to the ISA, the Python ISA simulator is the spec, and the RTL (under Verilator) and the physical card must each reproduce the simulator's output to the bit (README).

tools/validate.py compares a device's greedy tokens and logits against a CPU golden: the same Hugging Face checkpoint run in fp32 with openTPU's quantization applied. The golden multiplies the exact values the matrix unit uses and rounds activations to int8 per 128 values wherever the device rounds them. The device can be the ISA simulator, the RTL, or the card; --against pins a second device that must return identical tokens and bit-identical logits. The tool reports the largest difference in ulps and the first step that differs (validate.py).

A CPU golden cannot round exactly as hardware does — sums, norms, and exponentials differ in the last bits, and once one int8 value rounds the other way the error propagates through later layers. So the project measures a floor: the distance the golden sits from itself after a one-ulp nudge, and treats a healthy device as one that sits about that far from the quantized golden. On the ISA simulator, the device's mean KL from golden runs 0.84–1.16× that floor in the tool's own measured bounds; a run passes when top-1 agreement is at least 80% and mean KL is at most 3× the floor (validate.py hard-codes these as MIN_TOP1 = 0.8 and KL_RATIO = 3.0).

On the card (build 84989047, 2026-10-07), six runs of Qwen3-0.6B, LFM2.5-230M, and Qwen3.5-0.8B — in int8 and in 4-bit with an int8 head — returned the simulator's tokens and bit-identical logits on every prompt, 0 ulp over 93–128 steps. The card's mean KL from the golden was 0.95–1.25× the floor, with 95.3–99.2% top-1 agreement (the original write-up).

The project names the gap itself: validate.py covers Qwen3-0.6B, LFM2.5-230M, Qwen3.5-0.8B, and Gemma 4 E2B, but not the mixture-of-experts models — the source says plainly, "The MoE models are not supported" (validate.py). Those still match the simulator bit for bit in the card's own decode-loop runs, just not through the golden comparison (README). The author also concedes on HN that there are "some bugs/non standard behavior regarding very small or very big floating point values," which he claims are "proven harmless for LLM inference" — self-reported, and worth treating as such (comment).

Performance

Decode is bound by DRAM, not compute: LiteDRAM reads 82–85% of the DDR3-1066 peak, and the current production image (build B) pushes several models to 91–94%, decoding LFM2-2.6B, SmolLM3, and Phi-4-mini 8–9% faster than the previous image (README). Device decode throughput, measured on the card:

• LFM2.5-230M: 59.0 tok/s int8, 85.8 tok/s 4-bit
• Qwen3-0.6B: 21.6 tok/s int8, 31.3 tok/s 4-bit
• Qwen3.5-0.8B: 17.6 tok/s int8, 24.5 tok/s 4-bit
• Qwen3.5-2B: 8.02 tok/s int8; LFM2-2.6B: 6.05; SmolLM3-3B: 5.00; Phi-4-mini (3.8B): 3.99; Gemma 4 E4B (mixed int8/4-bit): 3.78

(README) The 4-bit format — FP4 with two-level block scales, 4.25 bits per weight, int8 LM head — cuts bytes per token by about a third and raises decode speed 40–45%, at a per-model perplexity cost the docs report. With logits streamed back while the card runs, 4-bit decode reaches 89.5 device tok/s on LFM2.5-230M and 6.69 on Phi-4-mini (README).

Models larger than the card's 4 GiB run with experts streamed from host storage into per-layer DRAM slots; the card routes every token and computes every expert. Slot hit rate decides everything: LFM2.5-8B-A1B hits 98.5% of uses and streams 5.2 MB per token, reaching 10.6 tok/s, while Qwen3.5-35B-A3B hits only 62%, drags 153 MB per token across PCIe at 1.41 GB/s, and lands at 3.95 tok/s (README). During decode the host adds just 0.17–0.30 ms per token on the author's omarchy machine (0.45–1.3 ms on the older i7-4790 host) (README).

The project names its own limits: the last few percent of DRAM efficiency, a 133.33 MHz clock that closes with only 0.032 ns of slack, and prefill still bound by the matrix unit's multiply rate (original write-up).

Why it matters

The result worth copying is not the throughput — this is a DDR3-bound Kintex-7 and it is not competing with GPUs, as the author says outright: "GPUs are faster, but you can't make your own arch on GPUs" (comment). What changes what you build is the loop: agents produced the entire hardware–software stack, and the repo ships the harness that independently established the stack is correct — an executable spec, bit-exact equivalence between simulator, RTL, and silicon, and an honest numeric floor for the parts (CPU-vs-device rounding) that cannot be bit-exact.

Everything except the card runs on a laptop: pip install -e ., pull a model from Hugging Face, and otpu-chat --backend isa chats on the bit-exact simulator; checking the RTL under Verilator against the simulator for three tokens takes about four minutes on a 16-core host (README, validate.py). If you want to evaluate whether agent-written hardware verification scales, the pass/fail harness — validate.py exits 0 or 1, --json writes every step — is the reference point, and the explicit MoE gap is the honest counterexample.

If the author's stated trajectory holds — DDR4/HBM2 FPGAs next (comment), and eventually an ASIC — the second-generation project aims to run "the inference of the models that are being used to improve it."

Caveats

• Every number above is self-reported in the README. As AICoder's write-up notes, there is no independent reproduction and no like-for-like GPU comparison yet. The strongest claims are device-vs-simulator bit-exactness; device-vs-golden comparisons are distance-based by design.
• The agent authorship is not independently auditable from the repo: the README states it and the author describes the process on HN, but the prompts, transcripts, and iteration history are not published, so "written by AI agents" rests on the author's account relayed by the original write-up.
• The MoE models are excluded from validate.py's golden comparison, and the author concedes non-standard FP behavior at extreme values (claimed harmless for LLM inference).
• Single-card scope: no chip-to-chip or multi-card scaling is demonstrated. The author himself calls chip-to-chip communication and KV-cache management "make or break for an AI ASIC" (comment).

Sources

• FeSens/openTPU — GitHub README
• FeSens/openTPU — tools/validate.py
• WPS syndication of the original write-up
• Hacker News thread (348 points, 405 comments) — including the author's comments on the recursive improvement loop, FPGAs as ASIC proof, DDR4/HBM2 plans, admitted FP edge cases, and chip-to-chip/KV limits
• AICoder: openTPU coverage and attribution caveat