neuwirth/levineuwirth.org
neuwirth
/
levineuwirth.org
1
1
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

auto: 2026-08-29T21:22:48Z [skip ci]

This commit is contained in:
Levi Neuwirth 2026-08-29 23:22:48 +02:00
parent 956a26b5af
commit b140ebea68
No known key found for this signature in database
1 changed files with 4 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
content/essays/verified-inference/index.md
View File

@ -61,9 +61,7 @@ VerInf instead bounds the **unexplained information** in an output stream: the
number of bits in the output that the committed model does not account for. If number of bits in the output that the committed model does not account for. If
the operator swapped in a different model, the outputs it produced would be the operator swapped in a different model, the outputs it produced would be
poorly predicted by the model it committed to, and the bound rises. A cheap poorly predicted by the model it committed to, and the bound rises. A cheap
substitution is therefore expensive to hide. substitution is therefore expensive to hide. Our certified statement is a conjunction of three parts held to different standards:
Our certified statement is a conjunction of three parts held to different standards:
- a **transcript anchor**, which binds the committed token streams to digests - a **transcript anchor**, which binds the committed token streams to digests
recorded independently at generation time. This ensures proof is about the run that recorded independently at generation time. This ensures proof is about the run that
@ -104,10 +102,7 @@ trust of the datacenter in which this takes place would be required.
## Results on record ## Results on record
These are the published single-chip results, and they predate my involvement. First, the single-chip results, which predate my involvement.
My own measurements — validation of the cost model, and the first figures from
Blackwell hardware — are in [What I contributed](#what-i-contributed) below,
separated there by what has merged upstream and what has not.
| | Llama-4-Maverick | Llama-2-7B | | | Llama-4-Maverick | Llama-2-7B |
|:---|:---|:---| |:---|:---|:---|
@ -121,11 +116,11 @@ separated there by what has merged upstream and what has not.
All on a single NVIDIA DGX Spark. The committed witness for the Maverick run is All on a single NVIDIA DGX Spark. The committed witness for the Maverick run is
roughly 7.2 TB, streamed at the working set rather than held. roughly 7.2 TB, streamed at the working set rather than held.
At 0.880 bits per token against a 202,048-token vocabulary, the proof accounts for about 95% of the information a At 0.880 bits per token against a 202,048-token vocabulary, the proof accounts for about 95% of the information a
token could carry. That is not "the model produced this"; it is "very little token could carry. That is not "the model produced this"; it is "very little
here is unexplained by the model that was committed." here is unexplained by the model that was committed."
## What I contributed ## What I have contributed thus far
Parallelizing the prover across GPUs is named future work in the paper, and it Parallelizing the prover across GPUs is named future work in the paper, and it
is the direction I am working in. It runs into an immediate problem: you cannot is the direction I am working in. It runs into an immediate problem: you cannot