Blog
Technical deep-dives, case studies, and engineering insights from the Richland team.
capy-flash runs a 125B MoE in full NVFP4 on two 16 GB GPUs at 100 tok/s — and documents the TP=2 partial-sum rounding trap
Apache-2.0 engine offloads routed experts to host RAM; the author shows FP16 transport payloads restore single-GPU numerical parity at no speed cost
ORL: diffusion LLMs get a learnable "tokens per forward" — 2,788 tok/s on one RTX 5090, 7,708 on 8× B200, fully open source
Together AI's on-policy residual learning plus the dlmrt engine make diffusion decoding competitive with autoregressive speculative decoding — and the release is fully open, with a bit-for-bit reproducible QwenDLM recipe.
OpenGCUA: an Apache-2.0 harness lifts all 8 frontier models it tested on OSWorld-v2 (+19.6 points, −53% steps) — but real-device runs show the model gap it hides
Alibaba Tongyi-MAI's open computer-use stack ships harness, SDK, and trajectories — no weights — and its own 4-task real-Mac results contradict the benchmark's equalizing story.
COUNT: a 797 GB read-only n-gram table in host RAM buys a 1B LM the loss of 57% more compute — zero learned parameters
UW/AI2's COUNT builds n-gram embeddings from precomputed corpus counts in host RAM, adds no learned parameters, and swaps tables at inference for training-free domain adaptation.
When the KV cache is slower than the GPU: a 124× p99 fix for context-parallel prefill
A production GLM-5.3 teardown finds remote KV hits slower than recompute — break-even is 2.53 GB/s, and the fix is sharding by rank.
Latent Compaction: the KV cache as a trainable network — 96.7% on RULER at 32× (logical) compression, code released
Tsinghua's THUNLP optimizes cache entries per document with 2,000 gradient steps of test-time training and frozen weights; the per-document GPU cost is nowhere published.
jitllm relocates a running model's layers between GPUs mid-reply: 8 ms pause, KV cache moves with the layers
A Go engine treats layers like OS processes: JIT kernels, a paged KV cache that travels with them, and an 8 ms handoff on T4s.
Delta Tokenizer: exact delta tokenization cuts agent-session P99 TTFT 9.5× under load, 23× when long requests tie up the front end
ASU's CPU tokenization front end reuses the previous turn's token IDs instead of re-encoding the whole session — ~40× per-turn speedup, 59% median TTFT cut inside SGLang, reference-identical on every checked output.
openTPU: AI agents wrote the RTL, ISA, and compiler for the chip that runs their own inference — bit-exact across ten models on a Kintex-7
FeSens' Apache-2.0 monorepo pairs a deliberately simple, no-cache accelerator with a verification harness that measures a one-ulp floor instead of pretending exact equality.
GraphTok: tokenization as a replayable CUDA graph — one Dynamo frontend feeds 56 GPUs of agentic traffic where the default feeds 2
KTH's preprint runs regex pre-tokenization and BPE merging on-device at bit-exact fidelity across five tokenizer families — all author-reported, and no code is released yet.
Activation steering as memory: facts fail, dislikes invert, and the "leanings transfer" claim failed its own pre-registration
A pre-registered study on Qwen3.5-2B/9B finds a gated activation memory is selective but stores facts as a word to say — and recalls a stored dislike as a like unless the opposite is subtracted at write time.
X-AgentRed red-teams personal agents by searching tool-dependency graphs, not prompts — 1.77× the harmful path rate of STAC, 2.99× the tool coverage
Meta's self-evolving framework builds a HarmFlow graph of effect→precondition handoffs, samples recipes backward from a target harm, and uses execution feedback to diversify and refine the search.