Accuracy Dropped From 93% to 7% — Cryptography Wants to Put a Dashcam on AI

Nathan Labenz had been looking for months for someone who knew both machine learning and modern cryptography. Daniel Kang, an assistant professor of computer science at the University of Illinois, is that person: he has been bringing zero-knowledge proofs to AI inference so a provider can prove it ran the model it promised, and a person can prove they are human, without handing over the inputs. The episode stays on cryptography rather than tokens. Labenz jokes that they will not be launching any coins.

Kang's long-term worry is simple. As generative models get better, proving you are a person currently means revealing more of yourself. Dating apps, he has heard, are among the biggest customers of services that photograph your face to confirm you are real. Those services upload the biometrics to a server. That is a surveillance problem as well as a privacy problem, and without cryptography he does not see a way around it.

Labenz folds in Worldcoin, the project that scans irises with a silver orb and tries to issue each person a unique ID. A square headshot stopped being enough around the time This Person Does Not Exist went online, so the checks keep escalating into more sensitive data. Kang wants the opposite path: prove authenticity without giving the sensitive inputs away. Cryptography, he says, "gives us a hammer that lets us bypass this fundamental trade-off between privacy and authenticity."

The other motivation is what Labenz starts calling model fraud. OpenAI will change the model you are being served — including ones you pay for — without warning and without a quality guarantee. Researchers at Stanford and Berkeley measured ChatGPT over time and found a substantial drop: on one task, accuracy went from something like 93% down to about 7%. As a customer you have no recourse, Kang says, because you have no idea what OpenAI is running.

The same cryptography that proves a person is human can prove that a provider ran a specific model, including the one that was running a month ago. Labenz adds a practitioner footnote: if you trust OpenAI, they now freeze weights on timestamps and give developers a window to revalidate before upgrading. If you do not trust them, that scheme does nothing. Plenty of newer providers have neither the freeze nor the reputation, and more reason to swap in a cheaper model.

The medical version is darker. A malicious attacker, even a state actor, could sit in front of a cancer-screening model and flip the output either way: tell a sick patient they are fine and delay treatment, or tell a healthy patient they have cancer and push harmful care. Today there is no way to tell whether a prediction came from the model that was certified by the FDA. Kang wants a proof that binds the provider to that model.

After a sponsor break, Kang states the property Labenz had been circling. Zero-knowledge proofs let you show that a process ran faithfully without revealing its inputs. For inference, those inputs are the weights the provider will not share and the prompt the user may not want to share. The salient part is not only that the computation ran. It forces the prover to stick to it. If the deal is model A, they cannot quietly run model B and pass off the result.

Kang's working analogy is a GPU that proves it ran a specific model. You still specify architecture and weights — his system currently takes variants of TensorFlow and a format PyTorch exports — plus the input, then you hand the bundle to the prover instead of a chip. From both sides it is meant to feel black-boxed: the provider produces a proof that it ran this model on this input, and the recipient verifies that claim. A new proof is required for every input.

The protocol has a time order. First the provider publishes a hash of the architecture and weights, a pre-commitment the customer can treat as inalterable. Then the customer calls the model, the provider executes and produces a proof, and the customer checks the pre-commitment plus the proof. Kang says the commitment is the easy piece: a cryptographic hash is a program you assume is hard to invert. That is what secures Bitcoin and billions of dollars of value. There is no strict mathematical proof that these functions are unbreakable, but decades of practice have not produced a break.

The wrinkle is entropy. Password-cracking works because common passwords have almost none; "password" is a high-probability guess. Salt the input with random bits and the hash becomes hard to reverse-engineer. Model weights can be low-entropy too, so Kang salts them the same way: append random bits, hash, and the commitment looks random. Some hash families are believed to be quantum-resistant. The assumption is still an assumption, just one the industry already bets real money on.

The specific tool is a ZK-SNARK, a form of zero-knowledge proof that can attest that an arbitrary computation ran correctly. The function Kang proves has two parts: hash the salted weights and reveal that hash, then run the model architecture on the given input and reveal the output. Because the hash is computed inside the proof, the provider cannot swap weights. The architecture is fixed in the function, the salted hash fixes the weights, and revealing the input fixes the prompt.

Labenz had pictured "arithmetization" as an arithmetic approximation of the forward pass, needed because models mix matrix multiplies with nonlinearities such as ReLU and tanh. Kang says the name is historical: proofs used to be built from arithmetic circuits, so you translate a Python-like function into a format the prover can ingest. For ML, that means turning architecture and weights into that format. At this step there is no approximation. It is an exact one-to-one mapping.

Nonlinearities become lookup tables, exhaustively precomputed at the quantized precision, the way old logarithm tables worked. Run that artifact and you get the same outputs as the quantized model. Labenz thought the tables might hold on the order of 10^8 entries. After quantization Kang puts the size closer to 2^15, hundreds of thousands or low millions of possible inputs. Quantization itself is not a validity requirement: the system can prove full floating-point arithmetic, it just becomes much slower. OpenAI, Google, and — Elon Musk has said — Tesla's cars already serve quantized models.

Almost every proving system Kang knows relies on a fact about polynomials: grow the number of variables and you can represent any computation. Labenz reaches for Taylor series. Kang prefers a reduction. NAND and NOR gates are universal; a multivariate polynomial can encode those gates by multiplying and adding variables; therefore a polynomial can encode any computation. The work then happens over a finite field — integers from zero up to a large number, wrapping around — with addition, subtraction, and multiplication.

The useful property is that two different polynomials, evaluated at a random field element, almost never match. Even a one-coefficient difference makes the values completely different. So a prover can commit to a polynomial and a verifier can check equality at a single random point instead of inspecting every coefficient. The secret in the constructions that work this way is a random large prime, which cryptographers call toxic waste, because nobody can reverse it. That irreversibility is the point of the prime-based scheme.

Labenz had worried that somewhere in this pipeline the representation stops being a faithful stand-in for the original forward pass. Kang says that complication appears when quantization is done badly, and that upcoming work repairs it. Think of the result as a fully faithful representation of the forward pass. There is no indeterminacy hiding in the proof itself.

Labenz's biggest misunderstanding coming in was that zero-knowledge proofs were probabilistic, with more compute buying more confidence. Kang splits the stack. An individual proof is unforgeable, an exact representation. If you want GPT-4's accuracy on a 100,000-example test set, proving every row is very costly, so you ask for proofs on 100, or 1,000, or 10,000 and treat that as a statistical estimate. The proof is the foundation. Sampling is a building you put on top of it.

The paper's rule of thumb is about 10,000 times more expensive than a normal forward pass. Prove one inference in 10,000 and you roughly double the compute bill. Labenz runs GPT-4 prices — a few cents per thousand tokens — times 10,000 and lands near $400 per verified call, on the order of a human arbitrator. Kang hopes for work in the next few months that cuts cost by orders of magnitude. Until then you would not verify every token; you would sample.

The user receives the model output and a proof, which Kang describes as a hash of the computation, and checks it locally with more finite-field arithmetic. Checking is much cheaper than proving: the prover has to work with the whole polynomial, the verifier evaluates it at a random point. The verification key specifies the layout of that polynomial. It encodes the architecture and the hash, not the weights. You could plug in your own weights and run it; the hash would not match, so you could not produce a passing proof.

Kang's analogy is Twitter open-sourcing its recommendation algorithm. You can read the PyTorch architecture and still cannot generate Twitter's timeline, because you lack the weights and the data. The verification key is that architecture plus the hash. Only the party that knows the real weights can emit a proof that matches the commitment. Labenz calls this the conversation's biggest update: two people can do it one-on-one, with no platform, reputation system, or regulator, if they will pay the compute tax. There is no way to cheat it.

Labenz has long thought smart contracts were limited by the word "smart." They auto-execute, but explicit code is a brittle way to handle real-world disputes, and people still want a place that can reason. As a GPT-4 red-teamer he tried basic mediation and found the model quite good at it, especially once multimodal evidence — photos, recordings — can enter the file. First-line AI dispute resolution starts to look plausible. Then it matters which model is running, and whether anyone can audit the execution when the stakes are high.

Kang likes the direction and names two blockers. Blow GPT-4's cost up by 10,000 and you are back near a human arbitrator. Both parties also have to agree on a specific model and on the outcome, and prompt wording changes the result. Contract disputes are full of human complexity. "I personally think that lawyers are going to be the last people to be automated away," he says, because people do not like a random machine deciding their lives. Technologically, he still puts cost as the main blocker.

How far can the tax fall? Upcoming results, Kang says, bring certain kinds of models down by something like 10 to 100 times. Transformers in the GPT-3/4 style are harder because of how the computation is laid out, so the savings there are smaller. Cryptography and polynomial evaluation are both moving every couple of weeks. His hopeful sketch is that in a couple of years the overhead might be around 100 times — still expensive, but in a regime where new kinds of low-stakes protection become possible.

Labenz's homely example is an old house and the odd plaster jobs it needs. Individual contractors struggle to build reputation; paying up front risks being ghosted. A roughly $10 recourse path would change that: photograph the wall with an attested camera, agree on a model, and let it decide whether the work matches the promise. Even with open weights, zero-knowledge still earns its keep. A smart contract cannot cheaply run standard code, so it needs a proof that the model ran as agreed, and you probably do not want interior photos of your house sitting on a public chain.

Deepfakes close the other loop. With existing technology, Kang says, you cannot tell whether an image is real or fake. The way around that is attested sensors: a camera that hashes and signs the bits at the sensor, physically next to where light becomes an electrical signal, before those bits go anywhere else on the device. Combined with time-binding tricks, you can be convinced a photo of a plaster wall was taken at a particular moment. That is also his alternative to walking up to an orb and trusting a company not to sell your iris information, which he says is apparently happening today.

If Apple, Google, and other phone makers play along, every future smartphone could carry an attested camera. You photograph your face and a driver's license, run a model on device, and reveal only a statement such as "I am a United States citizen over the age of 21." Weights can be known to both sides; the inputs stay with the person holding the phone. Consumers keep their biometrics. Service providers never store the sensitive data, which also shrinks their own breach liability.

Kang's picture of daily life is TLS. Recording this episode on Riverside, neither host nor guest thinks about the lock in the browser; HTTPS just runs. If proof costs fall, high-stakes inference should feel the same: an X-ray or CAT scanner signs at the device, the medical model attaches a proof, and a dispute can be audited later. Hardware makers still have to ship the signing units. Runtime cost need not hit users — you can bypass the signing channel and take a normal photo. Apple already ships a Secure Enclave on every iPhone; it is just not hooked up to the camera in a way developers can access. Kang's guess at the incremental manufacturing cost is one to ten dollars. He would like to talk to anyone working on a mandate.

Labenz notes that most photographs now come from a handful of phone makers, a sharper choke point than GPU telemetry. Kang's remaining preview is research: papers in the next few months that cut costs dramatically for certain models, and a method for auditing ML models in a very specific way. Until proofs are cheap, the practical moves are upstream of cryptography. Pin the model version in your code. Ask vendors how they will prove what they ran. And do not bet on after-the-fact AI content detectors. Source signatures, and original files from original devices, are the path Kang is willing to argue for.