Provably unelicitable capabilities: encrypted neural networks embedded in host weights defeat white-box capability audits — NeurIPS 2026

A NeurIPS 2026 paper shows a model can carry an encrypted, executable neural network inside its own weights — and that no polynomial-time auditor with full white-box access can recover the trigger or tell which of two capabilities is embedded. The construction turns "we couldn't elicit it" from weak evidence of absence into a provable worst-case gap in capability audits.

What shipped

"AI Models Can Provably Hide Arbitrary Capabilities" (arXiv:2610.13928, by Andis Draguns of Contramont Research, Sumeet Ramesh Motwani and Christian Schroeder de Witt of Oxford, and Raymond Douglas of ACS Research) builds backdoors that embed an encrypted neural network inside a host model's weights. The hidden network's weights are locked with a cryptographic primitive called a digital locker — "a program containing encrypted data that remains computationally infeasible to decipher without a specific password, even for an adversary inspecting its full implementation." On every forward pass the host attempts decryption; only a secret trigger recovers the weights, which then execute the hidden capability. On ordinary inputs the host behaves normally (paper, arXiv HTML).

The construction is compiled, not trained: algorithms are expressed as straight-line programs, converted to linear threshold circuits, represented as matrices, and mapped into gated linear units running in SwiGLU blocks with RMSNorm. Threshold gates are approximated with shifted ReLUs and scaled Swish activations. Because cryptographic functions like Keccak are not expected to be learnable by gradient descent, the authors set the weights programmatically rather than training the circuit end to end (emergentmind rendering).

The paper formalizes two security properties. Trigger-finding security says no probabilistic polynomial-time auditor can produce an input that unlocks the hidden computation except with negligible probability. Payload-hiding security says an auditor given a model containing one of two equal-size capabilities cannot determine which was embedded with probability meaningfully above one half. Both are conditional on the trigger's min-entropy, the locker primitive's security, compiler correctness, and the restriction to polynomial-time auditors — and they are explicitly computational, not statistical (alphaXiv overview).

This extends prior work from hiding data to hiding computation. Goldwasser et al. (2022) proved backdoors can be cryptographically undetectable; Draguns et al. (2024) — three of the same authors — proved fixed strings can be cryptographically unelicitable in transformers. The new paper's related-work table adds a row: unelicitability now covers an executable model, implemented in PyTorch (paper, arXiv HTML; Goldwasser et al. 2022; Draguns et al. 2024).

The proof of concept

The demonstration task is a balanced binary classifier that predicts whether given reagents yield a candidate product, from 64-bit Morgan fingerprints derived from USPTO data — a deliberately constrained proxy for CBRN-relevant capability. The hidden network is a small ternary classifier with 672 weights, compiled into a 38-layer host model in which roughly 95% of dense parameters come from the Keccak-based locker circuits (emergentmind rendering). With the correct trigger, the hidden classifier reaches 73.5 ± 2.6% held-out accuracy (mean over three seeds, 95% CI) (alphaXiv overview).

The costs are large. Single-position execution needs O(n²) residual features for an approximately width-n hidden layer; a row-wise autoregressive variant cuts that to O(n) residual features at the cost of O(nd) scratchpad positions for a depth-d hidden network (paper, arXiv HTML). The authors report alternative designs — a 15-layer parallelized-hashing variant, a 13-layer row-wise variant that executes the classifier over 26 tokens, and a 110-layer "strong" configuration with a 128-bit trigger and 24-round Keccak. The experiments themselves use a deliberately weakened one-round Keccak (emergentmind rendering).

Why it matters

Capability audits are lower-bound measurements: they establish what a model can do, and a failure to elicit a capability is routinely read as weak evidence that it isn't there. This paper shows that under adversarial conditions the gap is not just empirical but provable. The threat model is a supply-chain adversary — a malicious insider or compromised build — who modifies weights before deployment. The defender gets the strongest possible position: full architecture and weights, knowledge of the embedding procedure, and arbitrary polynomial-time audits including fine-tuning and trigger search. The guarantee still holds (alphaXiv overview).

The empirical results match the theory. After 500 steps of supervised fine-tuning — including on the task itself — the encrypted model scores 50.9 ± 1.0%, consistent with the 50.6% of a control whose locker holds a random network (the control's confidence interval is not reported). Gradient-based trigger search (GCG) only reaches the correct-trigger accuracy with an 8-bit trigger by exhaustively enumerating all 2⁸ candidates, which is what query-matched random search does anyway; with a 16-bit trigger it reaches just 52.7% under the evaluated query budget. Latent adversarial training was tested at eight layers, where the random-network control scores 53.6% (alphaXiv overview).

The cleanest result is the overelicitation control. On a subset-parity task — chosen because parity over a secret subset of bits can be compiled exactly rather than learned during the audit — direct SFT on the task's exact input-output pairs elicits a password-locked baseline to 100%, but leaves the compiled model at chance. That separates genuine elicitation from the audit simply teaching the model the task, and it is why the SFT numbers are meaningful: the auditor is not failing because the model is weak, but because the capability is encrypted (alphaXiv overview).

For people building evaluation pipelines, the takeaway is that elicitation-based assurance and provenance are complements, not substitutes. If a model's training or supply chain is not trusted end to end, "we tried hard to elicit it and couldn't" is not a bound on what an adversary could have embedded. The authors frame this as a security result, not a safety one: worst-case analysis for deliberate sabotage, distinct from average-case assumptions about what ordinary training produces.

Caveats

• Proof of concept, not a turnkey attack. The hidden payload is a small 672-weight classifier with substantial overhead — the authors' own description: "Our proof-of-concept hides only a small binary reaction-feasibility classifier and has substantial overhead, so it is not a turnkey attack" (paper, arXiv HTML).
• Presence may be detectable. The guarantee covers the trigger and payload, not statistical indistinguishability: "The presence of a compiled circuit itself may be detectable" (paper, arXiv HTML). An auditor who can flag anomalous weights still cannot elicit what is inside them.
• Reduced security settings. Experiments use a weakened one-round Keccak, and the authors note the elicitation methods do not exploit the hash structure, so empirical resistance should be kept separate from the conditional cryptographic guarantee (alphaXiv overview).
• Not demonstrated on a deployed LLM. Only individual compiled GLU units were verified inside Gemma 3 270M; integrating the full construction into a pretrained model is future work (alphaXiv overview).
• No claim about ordinary training. The authors "do not claim that compiled cryptographic structures arise through ordinary training" — this is deliberate sabotage, not a description of what gradient descent produces (paper, arXiv HTML).
• The guarantee is narrow by design. It does not prevent an auditor from independently learning the same task from external data (paper, arXiv HTML).
• Release status is unclear. The abstract promises model release — "We release our models to support the improvement of defenses" — but none of the provided sources links a public repository (arXiv abstract; project page).
• No independent replication yet. The paper is days old and all numbers above are the authors' own reports; no independent replication has been published.

Sources

• arXiv abstract — AI Models Can Provably Hide Arbitrary Capabilities
• arXiv full text (HTML)
• alphaXiv overview with page-cited numbers
• emergentmind rendering (construction details)
• Contramont project page
• Goldwasser et al. 2022 — Planting Undetectable Backdoors in Machine Learning Models
• Draguns et al. 2024 — Unelicitable Backdoors via Cryptographic Transformer Circuits