Not self-decidable: LLMs can't draw the boundary of what an agent verifier can check — and judge agreement certifies nothing
A NeurIPS 2026 workshop paper measures the routing decision every escalation scheme assumes away — with a vendor-provenance asterisk.
A paper accepted at the "Who Verifies the Agents?" workshop at NeurIPS 2026 reports that LLMs cannot reliably decide which rules an agent verifier can settle with a fixed check and which require a judge. The failure is worst on regulatory text and on a deployed agent's own rules, where models err together in the direction that never gets escalated. The single-author paper, "Not Self-Decidable" by Anthony Rhodes of Confidential Core AI, also reports that the obvious remedy — agreement with a reference judge — certifies nothing.
What shipped
The paper names the problem "self-decidability": before a verifier checks anything, something must route each predicate of each rule to either a fixed, interpretation-free procedure or a judge. "Logs are retained for twenty-four months" is settled by a few lines of code; "the disclosure is adequate" is settled by no amount of code. Where that line falls is the decidability boundary. When a team writes its own checks, the split is fixed up front; when requirements come from outside — finance, healthcare, law — the agent enforces rules it did not write, and the split recurs at runtime for every predicate of every rule on every action, with no human in the loop.
The measurement: 832 predicates from six corpora (EU AI Act, FINRA guidance, a deployed credit agent, IFEval, MT-Bench, and one the accessible text does not name), each labelled by four foundation models from three labs under four rubric variants and two perturbation instruments — roughly 22,000 model calls.
The headline results, all as reported by the paper:
- Agreement is near-perfect where the answer is obvious and collapses on regulatory text. κ is 0.92–0.99 on unambiguous endpoints and 0.20–0.50 on predicates decomposed from regulatory text.
- The errors run in opposite directions. Two models lean toward over-claiming decidability and two toward under-claiming; the split runs across labs and within one, and a model's direction flips by corpus. There is no common bias to subtract and no reliably conservative model to pick.
- The contested zone is the central mass, not the tail. On the 231 regulatory-text predicates, GPT-4o calls 41 decidable against Llama's 145, its set lying inside Llama's on all but 2; on the 73 agent rules the containment reverses, Llama's 48 inside GPT-4o's 62 on all but 1. That leaves 61% of regulatory-text predicates in the contested zone.
- On the deployed agent's own rules, models stop disagreeing and err together, over-claiming that a fixed check will do — the direction that never gets escalated, so the mitigation is silently switched off exactly where it was needed.
- The labels are confident, not hedged, and systematically unstable under meaning-preserving paraphrase of the same predicate.
Why it matters
Every escalation-based remedy for an unreliable LLM judge — escalate the hard cases to a human — presupposes that the escalation decision itself is reliable. The paper claims to be the first to measure that presupposition and finds it fails in the direction that is systematically injurious: an over-claimed predicate is never escalated.
The constructive answer is CoVer (corroborate-then-verify). Unanimity across independent models only nominates a predicate; admitting it requires the check synthesized for it to survive intervention — reading fields the agent cannot write, responding when they are intervened on, and holding under deterministic rewording of the record. Corroboration alone is not enough: as reported by the paper, it is 97–99% precise where models err in different directions, but only 83% on the agent's own rules, where they err together. The interventional gate rejects most of what corroboration wrongly admits, at a coverage cost the paper reports rather than tunes away.
The sharpest sub-finding is that the obvious alternative — agreement with a reference judge — certifies nothing. Agreement climbs from 30% to 77% across calibration bands while the genuinely decidable share does not move, because a judge drawn from the population under indictment ratifies the blind spot it shares. Agreement measures whether a check is reproducible, not whether a predicate is decidable. A pre-registered ablation adds a related result: a far more prescriptive rubric buys agreement without buying correctness.
The takeaway for anyone building agent guardrails: don't ask a model to route its own verification, and don't treat judge agreement as a certificate. The boundary has to be constructed — with checks that are executed, not labels that are asserted.
Caveats
This is a vendor paper with a provenance asterisk, and the numbers should be read as such:
- Single-author, non-archival. The paper is 21 pages (9 content pages plus references and appendices) and accepted at a NeurIPS 2026 workshop that is non-archival — no formal proceedings, though submissions are double-blind reviewed.
- Vendor provenance. Rhodes is at Confidential Core AI, which sells runtime-verification and agent-governance products. The conclusion — don't trust a model to draw the boundary, construct it with an external verifier — aligns with the company's commercial narrative.
- The full text is hard to verify. The arXiv HTML truncates mid-paper and the PDF resisted text extraction in our checks. The sixth corpus and two of the four models are unnamed in the accessible text.
- The deployed credit agent is unnamed, so the "they err together on the agent's own rules" result cannot be independently audited.
- No code or data released, and we found no independent replication or community discussion as of writing — the paper is days old. Every number above is author-reported.