Answering Questions Without Seeing the Data: Andrew Trask on the Privacy-Preserving AI Stack

Andrew Trask opens by dividing the talk into two parts: how privacy tools change a data scientist's workflow, and what mature versions of those tools could do to society. He then poses the central question: is it possible to answer questions using data we cannot see? The concrete comparison is between classifying tumor images and classifying handwritten digits. The first requires scarce, regulated data; the second can be downloaded and trained in minutes.

Trask explains that tumor data is rarely collected, hard to move, highly regulated, and controlled by a small number of sources. Acquiring it is likely expensive, which forces a researcher to find financing and a business plan before answering the scientific question. By contrast, MNIST-style digit data can be downloaded and used with a state-of-the-art training script almost immediately. That asymmetry, he argues, pushes the machine learning community toward accessible tasks rather than questions about dementia, diabetes, Alzheimer's, depression, or anxiety.

He notes that working on those high-impact problems often requires dedicating a portion of one's life just to gaining data access, whether by starting a company or joining a hospital. OpenMined, the community he leads, had a little over 6,000 people working to lower the barrier to privacy-preserving machine learning. PySyft extends major deep learning frameworks, including PyTorch, with privacy-preserving primitives, with the goal of making private datasets as easy to access as a pip-installable tool.

PySyft extends PyTorch through a torch hook that monkey-patches new functionality into the library. Trask introduces a second core primitive beyond the tensor: the worker, a location where computation occurs, such as a hospital data center. A worker exposes a limited, whitelisted set of methods so a researcher can run computation inside the data center without direct access to the machine. Calling .send on a tensor serializes it, sends it to the worker, and returns a pointer that still exposes the full PyTorch API.

When code executes through that pointer, it looks and feels local but actually runs remotely and returns another pointer to the result. Trask flags .get as the critical bottleneck: if a pointer can simply request arbitrary raw data, remote execution adds little real security. PyGrid, the platform version, lets a researcher search across many datasets inside a large hospital and receive pointers plus metadata such as row and column descriptions, collection methods, types, and value ranges. It can also return small samples, which may be human-curated or short snippets from the dataset, to support remote normalization and feature engineering.

Trask describes remote execution, private search, and sampling as the 80/20 first cut of privacy-preserving data science. They let data remain on the remote machine while giving enough context to start feature engineering and evaluate data quality. More difficult cases require the tools that follow.

Trask says a .get call on a private pointer should fail; instead the user must pass an epsilon to add the appropriate noise. Differential privacy, loosely stated, permits statistical analysis while making guarantees about the privacy of other records in a database. Sensitivity is the maximum amount a function's output can change when one individual is removed or replaced. Epsilon is the privacy budget, an upper bound on the statistical uniqueness allowed to escape from the database.

He illustrates the intuition with randomized response, a technique political scientists use for taboo behavior such as secretly being a serial killer. Each respondent flips a coin twice out of sight: heads means answer honestly, tails means answer yes or no based on the second flip. The true mean is averaged with a 50/50 coin flip, so the center of the distribution can be recovered while each individual retains plausible deniability. Local differential privacy adds noise before data reaches the statistician, while global differential privacy adds noise to a query's output and therefore requires trusting the database owner.

Trask warns that ordinary data anonymization is dangerous, using the Netflix Prize as his example. Netflix replaced movie titles and user identities with numbers, but each row remained a statistical fingerprint. Researchers, whom Trask recalls as being at UT Austin, scraped IMDb to build a comparable matrix and re-identified users with high accuracy by matching viewing patterns and timing. Similar re-identification has happened with medical records, and Trask says some companies buy anonymized datasets, de-anonymize them, and sell market intelligence to insurers.

Epsilon zero would mean only patterns that occur at least twice are allowed, because no pattern is unique to one person. Repeated queries consume the budget: three queries at epsilon 0.1 spend 0.3, so a project should receive an epsilon budget it cannot exceed. The loss of individual detail is acceptable because differential privacy can reveal that smoking causes cancer without revealing whether a specific person smokes. That mirrors the generalization machine learning practitioners want, since models should learn patterns that work across people rather than memorize one-offs.

Trask lists three weaknesses in the remote-execution-plus-differential-privacy stack. The data may be safe, but the model is put at risk when a valuable cancer classifier is sent to many hospitals to learn. A second problem is joining data across owners who do not trust each other and cannot agree on who sends data to whom. A third is trust that the remote machine actually performs the computation as instructed, which leads to secure multiparty computation.

Secure multiparty computation lets multiple people combine private inputs to compute a function without revealing those inputs to each other. Trask illustrates it by splitting the number five into shares of two and three, held by two friends; neither share reveals the number, and both shareholders must agree to decrypt it. The extraordinary part is that computation can happen while the number is encrypted: each shareholder can multiply their share by two and end up with shares of ten. Active security can also detect computation that a shareholder did not authorize.

In code, a tensor can be shared among Bob, Alice, and Teo, with each value split into shares and distributed among the shareholders. The interface remains PyTorch-like while cryptographic protocols run underneath, so machine learning researchers do not have to become cryptographers and cryptographers can improve algorithms that machine learning users inherit. Encrypted training and encrypted prediction become possible. Trask calls the combination of remote datasets, high-level latent patterns with differential-privacy guarantees, and multi-owner computation the minimum viable product for privacy-preserving data science.

The catch is computational complexity: secure multiparty computation sends a lot of information over the network. Trask cites a roughly 13x slowdown over plaintext for state-of-the-art deep learning prediction, which he calls inconvenient but not deadly when the machines are fast cloud instances. The field also lacks hardware optimization comparable to what Nvidia provided for plaintext deep learning. The theory for answering questions with unseen data exists, but it comes from different fields, so the engineering and adoption work is still ahead.

Trask distinguishes his approach from earlier work that trained on homomorphically encrypted data. That earlier approach could force future users to encrypt with the same key and added substantial noise, degrading privacy and utility. His preferred design encrypts both the model and the dataset during training, performs the same computations inside the encryption, and avoids accuracy degradation or dependence on one public/private key pair.

He defines federated learning as remote execution at smartphone scale: a phone downloads a model, trains locally at night, and uploads the improved model rather than uploading raw tweets or messages. Google pioneered this style with hundreds of millions of users, and Trask praises the work for paying down technical risk and publishing detailed papers. He separates product-oriented federated learning, with many ephemeral phones and a fixed model, from more exploratory federated learning in which private clouds host datasets and data scientists arrive with arbitrary questions.

Federated learning by itself is not a secure protocol, Trask warns. A federated model can memorize data and later spit it back out, so it must be combined with differential privacy or other protections. The ensemble matters, not any single technique.

Trask zooms out by comparing data handling to urban plumbing. In Edinburgh, cobblestone streets lifted people out of muck that ran down from buildings without internal plumbing, and people got sick as a natural consequence of the technology around them. He argues that society now spreads data everywhere through cameras, satellites, and services in a similarly unplanned way. The problem is not malice but infrastructure that has not caught up.

Building plumbing required tearing up a working city and running pipes through already constructed buildings, which must have seemed laughable and logistically enormous. Trask says the basic building blocks for privacy-preserving AI already exist and what stands between us and a different world is adoption, maturation, and good engineering. Google's large federated-learning rollouts are an early example of that path.

The first major payoff he highlights is open data for science. Trask recalls his Oxford adviser Phil Blount telling him that everyone works on models, but the biggest historical jumps came from new datasets or new ability to process them, citing ImageNet and GPUs, AlphaGo's synthetic data, AlphaStar's roughly 200 years of StarCraft, Watson's Wikipedia-derived structured data, and Deep Blue's large open chess dataset. Thousands of enterprises, millions of smartphones, and hundreds of governments already hold data that remains largely untapped because of legal risk and commercial viability.

A dementia researcher at Oxford illustrates the problem: every hospital has only about five cases, so the disease is sprinkled across many institutions rather than centralized in one cancer center. Private data science platforms could let those datasets be used together without moving them into one place. Trask calls this a repeatable business model: become the gatekeeper that keeps a dataset safe while letting others leverage it.

At airport security, a screener sees every object in every bag just to occasionally find something dangerous or illicit. Trask uses this as an example of information leakage: the question is whether a bag contains something dangerous, but the process requires access to the whole bag. A sniffing dog is more privacy-preserving because it reveals only one bit of information, and it does not speak English or any other language. That is Trask's model for single-use accountability.

A single-use accountability system lets an auditor answer only the question they claim to be investigating. Instead of giving a person a raw video feed with millions of bits about everyone in the frame, one could build an auditable machine learning classifier that surfaces only frames or objects matching the claimed concern. This reduces dual-use or multi-use risk and could make some forms of accountability possible that were previously too sensitive to attempt.

Trask draws on his earlier work in email surveillance for investment banks trying to catch insider traders. The bank might have hundreds of millions of emails, and the sensitivity meant very few data scientists could work with the data to improve the system. Privacy-preserving infrastructure could widen the set of people who can improve accountability systems without giving them raw access to private communications.

Messaging apps such as WhatsApp and Telegram encrypt a message on the sender's phone so that only the recipient's phone can decrypt it, letting a service provider operate without seeing the content. Trask asks whether machine learning, encrypted computation, and differential privacy can do the same for entire services. A doctor visit is a computation between the doctor's medical knowledge and the patient's symptoms, history, diet, genes, and heritage. The output is a treatment recommendation.

Trask describes structured transparency as combining input privacy from secure multiparty computation with output privacy from differential privacy. Encrypt the model and the patient's image, compute the function while encrypted, and decrypt the result only for the intended party. In a skin cancer prediction service, the model could classify melanoma and return the decrypted prediction only to the patient, so the doctor role facilitated by machine learning never requires anyone to see the medical record. Differential privacy is needed only when the result will be decrypted for an unknown audience.

In the Q&A, Trask says he does not know how far regulators are from requiring differential privacy, and that UK GDPR is not prescriptive about it, but the US 2020 Census is using differential privacy to protect census data. He expects adoption in waves, starting with enterprises because they already hold the data and have a commercial incentive to keep it scarce while letting others ask questions of it. Ultimately, he argues, individuals should set their own epsilon budgets, but that requires an accounting mechanism so that spending across different institutions is not double-counted, perhaps an IRS-like registry or data banks. OpenMined's early traction included a $250,000 open-source grant program from the PySyft team.

On recommendation systems, Trask says the biggest mistake is optimizing for engagement rather than holistic outcomes such as whether a person gets a good night's sleep or has more meaningful friendships. Those targets are sensitive, but they are exactly the kind of information a recommender would need to optimize for them. Privacy-preserving machine learning could give recommenders access to better targets without exposing the underlying private data.