DRY's sequence breakers cap its repetition penalty: 66% of locked loops with g=6 survive, and DRY-C closes the gap

A new preprint proves that DRY — the repetition-penalty sampler shipped in llama.cpp, ExLlamaV2, and text-generation-webui — caps its penalty for loops whose repeated unit never crosses a sequence breaker, so those loops can lock and run to the generation limit. On eleven open models, 66% of locked units with a 6-token breaker-free run stay locked under DRY (1% at 8 tokens), and the authors propose a count-based variant, DRY-C, that they report eliminates run-to-the-limit loops in all 720 structured-output generations they tested.

What shipped

DRY ("Don't Repeat Yourself") is a decoding-time penalty that targets repeated continuations rather than repeated tokens. For each candidate token, it finds the longest suffix of the current context that matches an earlier span and would be extended by that token; if the match length n exceeds an allowed threshold L and the token is not a sequence breaker, it subtracts λ·β^(n−L) from the token's logit. It was introduced in "Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time" (arXiv:2608.22761) and is implemented in llama.cpp (PR #6839), ExLlamaV2, text-generation-webui, and more recently as an example logits processor in vLLM.

The new paper, "When Do Repetition Loops Lock? Truncation Sampling, Bounded Penalties, and a Blind Spot in DRY" (arXiv:2610.24715v1) by Yichuan Chen and Xin Wang of Zhejiang Sci-Tech University, studies the exit rather than the entrance: given a span that has already repeated n times, when can the model still escape?

Along a loop the context is deterministic, so the probability that a loop survives m more repetitions under any sampler is a product of per-step continuation probabilities, computable exactly from one teacher-forced forward pass (Proposition 1). A position is "locked" when the loop token is the only candidate left — continuation probability 1.

Truncation is what locks positions. Under top-p with threshold π > 1/2, a position is locked exactly when the loop token's raw probability q ≥ π (Theorem 1); min-p locks when ln q − ln a > ln(1/β); greedy decoding locks when q > a. In the authors' finite-horizon tests, 80–100% of sampled units were locked through the last ten repetitions under top-p 0.9, versus none under pure sampling.

Penalties can unlock a loop, but only if they grow. For an additive logit penalty d on the loop token, locking persists whenever logit(q) − d ≥ logit(π) (Theorem 2). A bounded penalty merely raises the threshold at which locking sets in; escape is guaranteed under top-p only once the penalty outgrows ln(q/r°), where r° is the raw probability mass of tokens not yet in the context and therefore unpenalized (Corollary 2). Full-context frequency penalties grow fast enough to qualify (as long as the unpenalized mass stays bounded below); presence penalties and windowed frequency penalties carry no such guarantee.

That is where DRY's blind spot comes from. To protect formatting, DRY's matches may not cross sequence breakers — by default newline, colon, quote, and asterisk. For a loop whose longest breaker-free run has g tokens, the penalty on the loop token is at most λ·b^(g−1−a) (Proposition 4): on such loops DRY is a bounded penalty, and the lock persists no matter how many times the loop repeats.

The numbers match the theory. On eleven open models, 66% of locked units with g=6 stay locked under DRY, and 1% with g=8. In free generation with llama.cpp, every one of the 109 loops that appear with DRY enabled has g ≤ 7 — the loops that survive are exactly those whose longest breaker-free run is short.

The proposed fix is DRY-C: add a count term that penalizes how many times the same continuation has already occurred, growing exponentially with the number of verbatim recurrences up to a look-back window. The authors prove this penalizes every loop exponentially, breakers or not (Proposition 5), and report that with DRY-C none of 720 structured-output generations looped to the length limit, with task completion staying close to DRY's.

Why it matters

The blind spot sits exactly where loops hurt most. Under default settings, the paper's prose prompts loop in 0–7% of generations, but structured prompts loop in 16–21% — and sequence breakers are densest there. The paper notes loops occur inside reasoning, JSON, and tool calls, which is precisely the g ≤ 7 regime DRY leaves locked.

The tradeoff is structural, not accidental. The breakers exist to keep DRY from mangling chat templates and formatting — the DRY team's own evaluation found that removing sequence breakers damages formatting while lowering loop rates only slightly further. The new paper's contribution is to show that this design choice provably caps the penalty: DRY stops being able to push the loop token below the truncation threshold once the repeated unit is breaker-free.

For anyone running local inference, the takeaway is that DRY is not a loop guarantee — it is a loop guarantee for loops that cross a breaker. The fix is cheap: DRY-C is a decoding-time-only change, no retraining, and it keeps the breakers. If it survives replication, it would be a drop-in replacement for the penalty term in existing engines — but that is speculation until code and independent runs exist.

Caveats

This is a three-day-old preprint with no peer review and no released code for DRY-C. The 11 models are not named in the accessible text, the DRY-C formula and look-back window details sit in sections beyond the abstract, and "task completion stayed close to DRY's" is qualitative — the abstract gives no exact comparison numbers. The lock theorems are proven for specific sampler orders — top-k before top-p (the llama.cpp default) and temperature before truncation — and llama.cpp's own chain applies penalties before truncation (per PR #6839). All DRY-C results are author-reported and have not been independently replicated.

Sources

• New paper (abstract): https://arxiv.org/abs/2610.24715v1
• New paper (full text): https://arxiv.org/html/2610.24715v1
• Original DRY paper: https://arxiv.org/abs/2608.22761
• llama.cpp DRY PR: https://github.com/ggml-org/llama.cpp/pull/6839
• vLLM DRY feature request: https://github.com/vllm-project/vllm/issues/8581