Reasoning Without Logic: Yann LeCun on Teaching Neural Networks to Think

Asked whether neural networks can be made to reason, LeCun says yes without hesitation, since humans are a working example. The real question is how much prior structure has to be put into a network so that something like human reasoning emerges from learning. He argues that our usual models of reasoning are based on logic, which is discrete and therefore incompatible with gradient-based learning, which he strongly believes in.

He suggests this is part of why many computer scientists have viewed deep learning with suspicion. The math behind it is closer to cybernetics and electrical engineering than to the discrete math of computer science. Computer science, he says, involves compulsive attention to detail where every index must be right and algorithms can be proven correct, whereas nothing in machine learning is exact. He calls machine learning "the science of sloppiness".

Fridman asks what a network that builds on prior knowledge, creates new knowledge and generalizes beyond its training set would look like. LeCun says that if he had precise ideas, they would be building it already, though other researchers are working on exactly this. What you need first, he says, is a working memory: a subsystem that can store a relatively large amount of factual or episodic information for a reasonable time.

He uses the brain as a model, with three main kinds of memory. The state of the cortex fades within about 20 seconds. The hippocampus holds short-term memories such as the map of a building you just entered, and long-term memory lives in the synapses. A system that reasons needs something like a hippocampus, which is what researchers tried with memory networks and neural Turing machines, and which transformers have to some extent in their self-attention.

The second ingredient is a network that accesses this memory, pulls information back, computes on it and does so iteratively. A chain of reasoning, he says, is a process that updates your knowledge about the state of the world, so there must be a recurrent operation. Transformers, in contrast, have no recurrence and only a fixed number of layers, which limits the number of reasoning steps.

Fridman wonders whether this could emerge simply from scale. LeCun is skeptical: it is not clear how to read from and write into an associative memory efficiently. The original memory network may have had roughly the right architecture, but scaling it so the memory holds all of Wikipedia did not work well, so he says new ideas are needed.

LeCun describes a second, classical form of reasoning based on energy minimization. You define an energy function that rises when things go badly and falls when they go well. With a good model of your own body and the environment, you can find a sequence of actions that minimizes it, such as grabbing an object or walking out a door. In optimal control this is called model-predictive control.

Such an objective can include minimizing the number of collisions and the energy spent on the movement. He views planning as a form of reasoning, and speculates that humans' ability to reason may have come from ancestors who had to plan in order to hunt and survive winters. It is, he says, the same capacity.

Asked about expert systems and knowledge graphs, LeCun says logic representations, with variables, values and constraints expressed as rules, are too rigid and brittle. Early fixes attached probabilities to the rules, as in the Mycin-style systems of the 1970s, and led to Bayesian networks, graphical models and causal inference. He acknowledges a lot of interesting work there.

He sees two main problems. The first is knowledge acquisition: it is hard to reduce a bunch of data to such a graph, and the approach relies on humans to encode knowledge. The second is that manipulating symbols with logic is again incompatible with learning. Geoff Hinton has advocated for decades replacing symbols with vectors, patterns of activity across neurons, and logic with continuous functions, which are compatible with learning.

LeCun points to a paper from about ten years earlier by Leon Bottou, now at Facebook, called "From Machine Learning to Machine Reasoning". Its idea is that a learning system should manipulate objects in a space and put the results back in the same space, which is the notion of working memory. Fridman asks whether this might learn simple logic operations, and LeCun says quite possibly, adding that how much prior structure to build in is the subject of his debate with Gary Marcus.