Two GGUF checkpoints fix a Spark-X2.6 reasoning loop by editing a single output-head row
A minimum-norm edit of ~2,000–2,500 weights in the unembedding row cuts a "but wait…" loop rate by 58–93% with no training run.

Two GGUF checkpoints fix a reasoning-loop failure in the small Spark-X2.6 models by editing a single row of the output head — no training run, roughly 0.00006% of the 4B's weights. The diagnosis is the interesting part.
What shipped
The models are graydini/Spark-X2.6-4B-heretic-noloop-headrow-GGUF and graydini/Spark-X2.6-1.7B-noloop-headrow-GGUF (4B, 1.7B, both Apache-2.0, published 2026-11-14). They derive from the Spark-X2.6 base (XHToken) and, on the 4B, a "heretic" decensored finetune. The author describes the failure as a "but wait…" spiral: the model reaches the correct answer, then refuses to commit, re-derives the same conclusion in different words, re-lists its tools, and never acts or stops. In an agent setting that is fatal — the model knows the answer and still won't pull the trigger.
The fix is a surgical weight edit, not fine-tuning. On the 4B it changes 2,560 weights (one row of the tied output head); on the 1.7B, 2,048 (untied head, so input embeddings are untouched). Every other byte is claimed bit-identical to the upstream quant. The repos ship a CPU-only apply_edit.py, the delta files, a ggml re-encoder, and an edit_report.json with source/output SHA-256 hashes so the checkpoint can be reproduced offline.
Why it matters
Because the head is a linear readout — logit(t) = emb[t] · h on the tied 4B, output_weight[t] · h on the untied 1.7B, where h is the final-norm residual — the author could attribute the failure to a specific place. Activation analysis at the moment the model reaches a conclusion but before it loops showed the closure token </think> at rank ≈ 54 with log p ≈ −11.8 (P ≈ 7×10⁻⁶), while the restart token " But" was top at P ≈ 0.45. That ~11-nat deficit is a pure head-readout gap, not an MLP "loop direction."
An earlier attempt to ablate an MLP down_proj circling direction was "correlated-but-not-causal": ±sign symmetric, max +2.85 nats — too weak. The fix instead solves for a minimum-norm row delta, δ = Hᵀ(HHᵀ)⁻¹(Δ·1), that raises δ·h_p by ~14 nats at the conclusion states (‖δ‖ ≈ 0.2, roughly 15% of that one row's norm). Because the edit is applied only along the conclusion residual directions, it is meant to be conclusion-selective rather than truncating legitimate reasoning.
Reported loop-rate results on a controlled branch-point benchmark (4 prompts × 12 seeds, n=48 per model):
| Model | Before | After | Reduction |
|---|---|---|---|
| 4B | 0.750 | 0.312 | 58.4% |
| 1.7B | 0.875 | 0.062 | 92.9% |
The tool-ambiguity prompts dropped to 0/12 on both models; the harder math prompts improved but did not fully clear. Capability evals were roughly neutral on the 4B (mean −0.002) and slightly better on the 1.7B (mean +0.047), where GSM8K went 0.200 → 0.433.
This is a clean instance of a thesis building across several recent results: behavior is linearly decodable in the final hidden state and editable in the unembedding. The closest published precedent is HeadEdit (arXiv:2610.01170), which calibrates behavior through the unembedding matrix via a low-rank behavioral subspace — though it does a vocabulary-wide correction rather than a single-row min-norm delta. Related work on weight-space programming points the same direction.
Caveats
Everything above is author-reported from the model cards and edit_report.json; no third party has independently reproduced the numbers or the bit-for-bit claim. Several things keep this from being oversold:
- The author's own materials disagree. The Reddit title/description claims "up to 97% reduction" on the 1.7B, but the model card and
edit_report.jsonshow 92.9%. The 97% figure is unexplained. - Small samples. The loop benchmark is n=48 per model; capability evals are n=100 (MMLU 91, GSM8K 60). The 1.7B GSM8K jump of +0.233 on n=60 is a large swing that could be small-sample noise.
- The fix is narrow. The author states it targets the thinking loop only: agent tool looping can still occur on broader tool-use tasks, and arithmetic re-derivation termination is not solved. An EOS-row boost was tested and rejected because it regressed some tool/clean seeds.
- "Conclusion-selective" rests on four loop prompts. The selectivity claim is supported by a handful of early-vs-conclusion P(close) measurements, not a broad sweep.
- Novelty is unverified. The single-row output-head fix appears new, but the Reddit comment thread (potential independent discussion) was inaccessible, and no independent verification was found.
The takeaway: a single row of the output head — about 0.00006% of the 4B's weights — appears to encode much of the "commit and stop" decision, and a minimum-norm edit of that row suppresses a reasoning-loop failure with no training. That is a useful diagnostic pattern for anyone fighting agent loops, even before the numbers are independently confirmed.
Sources
- 4B model card — graydini/Spark-X2.6-4B-heretic-noloop-headrow-GGUF
- 1.7B model card — graydini/Spark-X2.6-1.7B-noloop-headrow-GGUF
- 4B apply_edit.py
- 4B edit_report.json
- 1.7B edit_report.json
- Reddit thread — SPARK 2.6 4b and 1.7b fixed a reasoning loop
- HeadEdit (arXiv:2610.01170)
- Sparks of Weight-Space Programming