Self-Retrospection Distillation: turning RLVR's dead rollouts into training signal — 0.0% to 60.6% success in a 2B all-failure regime
Salesforce AI Research's preprint reports up to 24.2pp gains when SRD is added to RLVR and self-distillation baselines across 10 agentic tasks.
Salesforce AI Research posted a preprint proposing Self-Retrospection Distillation (SRD), a training objective that turns the rollouts that group-relative RLVR normally throws away — groups where every rollout earns the same reward — into a learning signal. The authors report that SRD complements RLVR and self-distillation baselines with gains of up to 24.2 percentage points across 10 tool-integrated reasoning and long-horizon agentic tasks, and a 0.0% → 60.6% success jump in a 2B setting where 98% of rollout groups are all-failure, at the same rollout budget.
What shipped
The paper, Self-Retrospection Distillation: Turning Post-hoc Experiences into Prior Foresight (arXiv:2610.08077v1, submitted 6 Oct 2026), is a 46-page preprint from Salesforce AI Research with UC San Diego, Texas A&M, and Stanford. It introduces an approach the authors call prospective learning: instead of using post-hoc trajectory experience to supervise the agent's actions (as RLVR and self-distillation do), SRD uses it to supervise foresight predictions made before acting.
The failure mode is familiar to anyone training agents with group-relative objectives like GRPO. Advantage is computed relative to the other rollouts of the same task: when all G rollouts in a group receive identical rewards, the normalized advantage is zero for every rollout, so the group contributes no gradient. The paper notes that standard practice discards such uniform-reward groups entirely and resamples until a discriminative batch appears. On hard agentic tasks that resampling loop is the problem: the authors report that across the model scales they tested, 37–98% of rollout groups were reward-uniform — dead weight for RLVR.
SRD stops discarding those trajectories. A completed trajectory, even a failed one, reveals knowledge the task required and the pitfalls the agent fell into. SRD distills that privileged hindsight into "trajectory-blind foresight" of the same policy: a structured prediction, made from pre-interaction information alone, of what the upcoming interaction may require and where it may fail. The model generates this foresight via a differently-prompted call to itself before acting.
The training step then pairs that pre-interaction prediction with a hindsight prediction conditioned on the completed interaction and aligns their token-level distributions on the student's own rollout, using a stop-gradient self-teacher. As the paper puts it, "reward thus decides which lesson a trajectory teaches rather than whether it teaches one."
Two properties make this cheap to adopt. First, foresight is a training target only — the paper states it "need not be explicitly generated at inference time," so there is no serving-time overhead. Second, SRD is an auxiliary objective layered on top of RLVR and self-distillation algorithms, so it composes with existing pipelines rather than replacing them.
The headline numbers, all author-reported from the abstract:
- Up to 24.2 pp gains when added to RLVR and self-distillation baselines, across 10 tool-integrated reasoning and long-horizon agentic tasks.
- 0.0% → 60.6% success in the 2B setting, where 98% of rollout groups are all-failure, at the same rollout budget as the RLVR baseline that ends at 0.0%.
Why it matters
Vanishing reward contrast is not an edge case in RLVR for agents — it is the main event. Long-horizon tool use produces mostly-failed rollouts, and when a group is uniformly failed, group-relative advantage is identically zero. Teams respond by resampling, which burns rollout budget, or by engineering reward shaping to manufacture contrast, which is fragile. SRD is a different lever: it makes the failed trajectories themselves informative, so the signal no longer depends on contrast existing in the first place.
For engineers, the practical shape of the contribution is a composable auxiliary loss with no inference cost. If the numbers hold up, it is the kind of change you can bolt onto an existing GRPO or self-distillation run — no new verifier, no new environment instrumentation, no change to serving. The 2B result is the one to watch: in the regime where RLVR flatlines at 0.0%, SRD still extracts signal from trajectories that standard practice would have discarded — exactly the regime where small models trained on hard agentic tasks stall.
Caveats
This is a one-day-old preprint with no independent replication, and the numbers above are the authors' own claims. More importantly for verification: at the time of writing, the full text is not accessible in readable form — the arXiv HTML version returns 404, and mirrors carry only the abstract. That means the per-task tables, the exact SRD loss, the model families, and the task names behind the "10 tasks" and the 24.2pp / 60.6% figures could not be checked against the paper's experiments section. Treat the specific numbers as claims pending a readable full text, not as established results. No public code repository was found at the time of writing.