Inside AI Models: Neel Nanda on Interpretability and Safety

Hannah Fry talks with Google DeepMind’s Neel Nanda about interpretability: methods for studying how AI systems process information and how those methods might support safety. The analogies to thought and neuroscience describe research tools, not proof that a model has human experiences.

Nanda describes scientific curiosity and safety as motivations. His expectation that human-level AI could arrive within a decade or two is a forecast, not a demonstrated timetable.

Mechanistic interpretability aims to connect internal model computations with behavior. Researchers disagree on how completely a complex system can be understood, and Nanda treats comprehensive understanding as an ambitious goal.

Nanda compares a reasoning trace to a scratchpad: it can provide useful clues and may reveal mistakes or undesirable behavior, but it is not a guaranteed faithful account of all internal computation. Future models may also learn to conceal information.

The discussion distinguishes examining inputs, outputs and reasoning traces from inspecting internal activations. Nanda introduces probes, which look for a chosen concept, and sparse autoencoders, which attempt to separate useful features in activation data.

A geometric analogy helps explain how directions in activation space can correlate with concepts. A detected direction is an analytical signal, not automatically a complete or causal explanation of what a model is doing.

Sparse autoencoders identify patterns that may be easier to inspect than raw numbers. Nanda notes their limitations: the features found depend on training data, and important concepts can be missed.

For safety auditing, the researchers want to detect misuse, hidden objectives or test gaming. Nanda describes an example in which both sparse autoencoder and black-box approaches helped uncover suspicious behavior; success on a specific case does not prove universal detection.

Looking ahead, Nanda hopes for cheaper monitors and stronger model audits, while urging realistic expectations about complete understanding. Interpretability is one layer of an evolving safety approach, not a guarantee.