Deep Learning Is a System 1 Machine: Daniel Kahneman on What AI Still Lacks

Lex Fridman opens by asking whether the features and bugs Kahneman has described in human thinking are useful for building AI. Kahneman answers that what is happening in deep learning is more like a System 1 product than a System 2 product. Deep learning matches patterns and anticipates what will happen, which makes it highly predictive. In his framework, System 1 is the fast, instinctive mode and System 2 is the slower, deliberate one.

What it lacks, he says, is the ability to reason, and there is no System 2 there. He adds that it has no causality and no way to represent meaning or real interaction. Until that is solved, he says, what can be achieved is marvelous and exciting but limited. Fridman finds it helpful to think of current advances in machine learning as essentially System 1 advances.

Kahneman says what has impressed him most about AI is the speed of progress. He points to the move from solving chess to solving Go, and the jump from AlphaGo to AlphaZero, which he calls bewildering. Many problems can be solved that way, he says, but some need something else, such as reasoning.

He raises one of the real mysteries, citing the psychologist and AI critic Gary Marcus. Humans learn quickly, and children need two or three examples, not millions. That suggests something must be built into a machine to make it ready to learn fast, such as expectations about the world. Kahneman says this is still unsolved as far as he knows, though he is sure DeepMind is working on it.

Fridman notes that people are trying to get neural networks to reason, and that temporal causality is out of reach for almost everyone. Kahneman observes that early AI tried to model reasoning with moderate success, but reasoning alone did not get far. Deep learning has been much more successful in what it can do, so the open question is whether it is approaching its limits.

Kahneman says he is with the majority who think networks will hit a limit. The discussion turns to Yann LeCun, who is described as a pragmatist: we can see one or two mountain peaks ahead, and there may be a few more after them or thousands. Kahneman later says LeCun's answer that we do not know how many mountains there are is a very good one, and he finds the people actually doing the work to be fairly realistic.

Kahneman says translation systems do a very good job but don't know what they're talking about. For that, he suggests, an AI would need sensation and contact with the world, and perhaps something resembling awareness, so that words connect to perception or action. Fridman calls this grounding in physical space. Kahneman says he is not sure a body is needed, but a perceptual system seems essential for accumulating knowledge, and a paralyzed human brain can still learn a great deal.

Fridman then raises active learning, the ability to play with the world. Kahneman says much of what we learn is how to anticipate the outcomes of our actions, as babies do when they connect hand movements to what happens. He points to how quickly people learn to operate artificial limbs, which suggests the brain is built to learn the world through action.

Fridman, who works on autonomous vehicles, asks how hard it is to model pedestrians well enough to predict whether they will cross. Kahneman is fairly optimistic, because every vehicle's learning feeds one gigantic system, a multiplier humans lack. Vehicles will make mistakes, but so do people, and drivers already anticipate how others react, for example when merging into a roundabout. Fridman counters that pedestrians are currently treated as things that cannot be hit rather than as agents in a game-theoretic interaction, and that nobody has seriously tried to solve that dance.

Kahneman describes how he crosses the street: he looks the driver in the eye, a signal an autonomous vehicle would need to understand. Fridman adds from hours of video that people also look away as they step off the curb, telling the driver they are committed, like a game of chicken. Fridman suggests a machine might anticipate without understanding, as in Go, but Kahneman suggests that for this a model of the human mind is needed, which makes the problem a lot harder.