Prediction is not a decision: Michael I. Jordan on markets and the unfinished AI revolution

Lex Fridman introduces Michael I. Jordan as a Berkeley professor cited more than 170,000 times, and as the mentor of people who now define the field, among them Andrew Ng, Zoubin Ghahramani, and Ben Taskar. Yann LeCun's nickname for him is the one Jordan prefers: the Miles Davis of machine learning, a musician who reinvented himself so often that even loyal listeners were left scratching their heads. Jordan accepts the metaphor because Davis, in his telling, was a real explorer with a coherent story. The story he wants to tell about this era is not the one on the posters.

Put on a historian's hat, he says, and what is happening now looks like chemical engineering growing out of chemistry, or electrical engineering growing out of electromagnetism. In the 1930s and 1940s there was chemistry, fluid flow, and mechanics, but not yet a discipline whose job was to build factories that made chemical products viably, safely, and at scale. Some of those factories worked, some were not viable, some exploded. In parallel, a field with its own theory and practice came into being. Maxwell's equations already said, in some sense, everything about electromagnetism; someone still had to figure out circuits, modules, and how to move electricity from one point to another without burning the house down.

Machine learning, if it is a field at all, is that proto-form of engineering, built on statistical and computational ideas from previous generations. The things being built are systems that bring value to human beings, use human data, and mix in human decisions. The engineering side of that, Jordan says, is still ad hoc. The dreams of putting thought in a computer may be real, but they are five hundred years out, like Greeks sitting around saying it would be neat to get to the moon. We have no clue how the brain computes. On the scientifically interesting questions of our era, he says, we are worse than the Greeks.

Fridman asks him to linger on the brain, because Jordan is unusually blunt about how dark the room still is. Jordan says he is not unique in that view: talk to people who study real synapses and real neurons and they will agree it is a task of centuries, built slowly. The signal itself is not clearly electrical or chemical. It is a soup of ions, proteins, and cells. Look at an electron micrograph of a single synapse and it is a city of its own, one little node on a dendritic tree that is already an electrochemical maze, with spikes, voltages, and proteins carrying traces down toward DNA.

Our talk of brains as economies, immune systems, or stacks of arithmetic is metaphor, and metaphor is not neuroscience. It is, he says, the Greeks speculating about how to get to the moon. He says this strongly because young people are told we are on the verge, that the work is brain-inspired, that breakthroughs sit on the horizon. Some of that overselling, he adds, comes from labs that need money. Computational neuroscience has done too much of it.

Asked about Neuralink and electrodes that read and write electrical signals without first knowing the principles, Jordan is polite and unconvinced. He would expect modern neuroscience to help with Alzheimer's and other brain disease long before it explains thought; we still throw lithium at mood and mostly do not know why it works. Hooking up electrodes to find patterns that move a cursor is fine, and he can imagine medical uses and even a business. Inferring a city's microeconomy from satellite emissions when you lack even macroeconomic concepts is the better picture of where we are. From there to algorithms that tie the brain deeply to a computer is, he says, not for our generation and not for this century. He does not agree with Elon Musk.

Fridman asks whether any breakthrough in five or ten years would make Jordan sit back and say wow. There will be breakthroughs, Jordan says, but they will not be demos. A computer that calls a restaurant, pretends to be a person, and even inserts coughs as a public-relations flourish is not a scientific advance, however hard the engineering and however useful the eventual product. He does not want to diminish the people who ship those systems. He wants to distinguish that work from an engineering discipline that has principles, some scientific understanding, and a predictable way of giving value. A young Kolmogorov, he says, would not be wowed by the stunts coming out of the big companies.

He also dislikes the habit of ranking theoretical breakthroughs above practical ones. If he had to point at a figure of the era he might pick an Edison rather than a Kolmogorov, and he allows that Musk sometimes looks more like the former — while also leading people astray when he talks about things he does not know. Deep understanding of language, of the kind in the conversation they are having, will not happen in our lifetime; you can fake it by replaying sentences humans already used. The breakthroughs he will grant are Google, Amazon, and Uber: value brought to people at planetary scale, in new ways, on data flows. They are slightly broken because no engineering field yet holds economic value, data, privacy, and externalities in one frame. Looking back a hundred years, he thinks that emerging field will be this era's breakthrough, the way electrical and chemical engineering were for the last century.

He resists calling any of this AI. John McCarthy, almost as a philosopher, asked whether thought could be put in a computer and tried to write logical formulas that would do it. That is a reasonable project. It is not what this era is doing. McCarthy needed a new name because cybernetics already existed. Norbert Wiener had packed control theory, signal processing, closed feedback, data, and some notion of intelligence into one language; McCarthy wanted to say he was not part of that show. As a title for movies it was inspired. As a name for a sober academic discipline it was terrible, because it promised something we cannot deliver. "We understand artificial perhaps, but we don't understand intelligence."

What the work is actually about, Jordan says, is large collections of decisions under uncertainty by large collections of entities. Pattern recognition matters because it serves those decisions. Expert systems, search, and knowledge representation can sit inside that picture if they help. We do not know what intelligence, abstraction, or human-level reasoning are, and we are not building those things because we have no clue. We are trying to make consequential decisions — a medical operation, a drive down the street — with computers, data, scarcity, and other people in the loop. The goal is working systems at a planetary scale we have never seen. Intelligence is a buzzword that will get attached to the result. It is not the goal.

The most interesting disagreement with Yann LeCun, Fridman asks? Jordan calls LeCun an old friend. They share a build-it-and-see-if-it-works temperament, they both speak French, and they do not disagree about much that is fundamental. The difference is emphasis. LeCun has pushed pattern recognition and prediction as far as they will go, including the thought experiment of perfect prediction. Jordan thinks that frame is far too limited. We will never have the data to know what question Fridman will ask next. Most of the day is spent in situations nobody anticipated, and in those moments you have to think about what you actually want, what a blocked street costs you, what the risks are, and what other people are about to do.

Current prediction systems, he notes, often give no error bars at all. He would not take a neural net's output on a consequential medical question even if it had every heart-attack record ever collected and beat every doctor he had ever seen. He would not trust a number that crossed 0.7. He would want what-if questions, causal alternatives, data nobody thought to gather, and a conversation with a physician about the gaps. If you say prediction is everything, you miss that stack. Prediction plus decision-making is everything, and both halves are equally important.

The field has talked about prediction, at the cost of forgetting that decisions are where lives are at stake, risks are taken, more data must be gathered, and an economy sits around the choice. Jordan says he is not the only person in that smaller tribe, but they are not the ones most people talk about. Walk into Amazon and, in his estimate, half the people are working on decision-making and the other half on pattern recognition. Lab data sets are a constrained setting. Consequential decisions are the mess of the real world, touching human beings at scale, under market forces. Real prediction, he adds, is not possible from a frozen data set alone. It has to live inside a strategy for what to gather next.

Asked for a picture of decision-making at the scale of millions, Jordan reaches for a thought experiment he can actually think with: a music market. Strictly speaking, he says, there is not one in the country. There are record companies that prop up a few musicians, make them superstars, and make a great deal of money. There is also a long tail of people, especially in hip hop and Latin music, who make extremely good songs on a laptop on the weekend, hold another job during the week, and put the files on SoundCloud. The bits get streamed. The economic information is lost. Walk a city and the headphones on young people are mostly not playing famous catalogs or old songs. They are playing the latest thing, often made by a sixteen-year-old who will never make a career of it.

Pipelines like Spotify take a cut from subscriptions or ads and then sprinkle a little of the money on a few highlighted names, which simulates a market without being one. A real market would start with data the creator can see. A dashboard, a map of the United States, last week's listens, transparent and auditable so that someone in Providence who sees ten thousand real plays will believe the number, invite a show, and tell those listeners the artist is coming. Do that right, Jordan says, and a trip might be worth twenty thousand dollars; three trips a year and a career starts. In that picture AI creates jobs. It does not take them away. Once producers and consumers are connected, the musician can haggle over a wedding gig, get income toward a hundred thousand dollars, and never need to be a millionaire.

The real value of music, he argues, is the personal connection, down to a kid who wants a T-shirt with a signature on it. The listening page should have a button; the merch arrives the next day; no advertising is required. The buyer is happy, the musician is paid, and a platform that takes a 5% cut can be a sound company. It sounds easy and is not. Two-sided markets at this scale have not been built before. There will be bad actors, data used the wrong way, and failures to deliver value, the way anyone who has run a large auction or matching service already knows. Silicon Valley cannot top-down hire the sixteen-year-old or buy the culture. Netflix can write checks to film-school graduates; American music, Jordan says, was not mainly a rich-white-people project. The technology has to partner with the culture rather than purchase it.

Jordan's favorite recommender, historically, is Amazon's book list. He disliked the company at first because local booksellers went under; he has since accepted that more books are sold and read, and that some shops came back. When he finally bought a few titles, the site recommended others he would never have thought of, he bought those too, and he still browses there. What he likes is the feeling of learning from peers, indirectly, with an algorithm that tries to surface what is in the data rather than impose a curriculum. Shirts and restaurants, he notes, are much harder. Blend recommenders with other economic machinery and the research is still wide open.

News, politics, YouTube, Twitter, and Facebook News are, in his view, not first a ranking problem. What is broken is monetization by advertising. Google and Facebook went back to the television playbook: nobody wanted to pay for the signal, so ads filled the gap. Google's little boxes on the right did not look economically interesting; they became a fortune, and the company, doing so well, did not think hard enough about a producer-consumer market sitting beside the ad market. The producer is the person who made the clip or the site and could make more if demand were visible. Influencers are a late glimmer of that market. It should have been designed twenty years ago, in parallel with ads. Facebook inherited the same model.

Once the money is in click-through, the company is incentivized to get more clicks, and when the system starts to break it reaches for smarter AI to sort good clicks from bad. Jordan finds that pretty hopeless. Fake news rides on top because the revenue is click-through rate. The better fix is the business model: are there producers and consumers whose direct connection is valuable enough that people will pay? It does not have to be a micropayment. If he is going to Mumbai for two days with no idea what to do, he does not want a wall of advertisers. He wants to broadcast the trip and have someone relevant — his age, someone who knows him a little — make a short video or a two-page note on what to do this week. He would pay a hundred dollars, not a fraction of a cent. The company takes 5%. Behind YouTube, people who could make more of those guides would make them without being told to.

He does not want zero advertising. A world with none would be worse, because ads are a signal that someone believes in a product enough to spend real money. An MIT student who has built a genuinely better vacuum cleaner will not show up in the recommender; algorithms will not find it. A costly ad can put it in front of people. That signal is now gone, drowned in inventory sold for pennies. Advertising currently annoys people more than it informs them, and societies do not stick with things that annoy. A serious company would lower ads without killing them, raise real producer-consumer payments, and let the 5% grow enough to replace the bad ad revenue. Amazon, where Jordan spends a day a week to learn, already lives in markets of packages and sellers. Google hovers in between. Facebook, he says, has not thought about the producer-consumer link at all. United Masters, where he sits on the board, had that model from day one: more than a hundred thousand artists, and NBA clip music that puts a kid's song on the league's site.

Fridman loves recommenders and would hand them everything about himself in a setting he trusted. Jordan does not want to be recommended from his browsing history, or from facts a company knows that he never offered. The unknown about him is the interesting part. That morning, still sleepy, he had clicked a story about the Queen of England. He does not care about the Queen of England. It was clickbait, it looked funny, and now it is in the trace. Any reasonable system will treat that click as interest. He does not want his life heading that direction.

The industry model, he says, has been that if you collect enough digital exhaust you will figure people out. Maybe you could figure out one very public person. Five hundred million people, no. Humans are rich and complicated, with quirks and loves that leave no sign in the past until they arrive — you fall for them. He does not want a company anticipating that. He wants a forum, a market, a place he can walk into where, by hook or by crook, something happens: Chilean music playing on the street that he never knew he liked. Create spaces where creativity and style come forward. Do not use AI to figure someone out and then put them in a world you have already solved. The current path, including anonymous comments that hurt people, is the wrong one for the whole technology. Let humans rise up. Do not try to replace them, anticipate them, or predict them; the little successes will make things worse.

Could a recommender talk to you the way a friend does, instead of watching from the shadows? Jordan is not against an assistant. He does not want a large company to know the context of his home, but he would take technology that helps. Alexa, done right, is still more a research platform than a product. If he says the water is running in the garden and the system infers that his wife should be told when she gets home, that sliver of reasoning would make the day easier. It would not be a wow moment. It would raise happiness a little. He does not want Alexa to know he is lonely. He wants control and transparency, not a designer who has worked it all out in advance. Microsoft under Satya Nadella, he thinks, is pivoting toward being a trusted old uncle, and that is the better business model: technology that gives people more control over a rich set of interactions, not just over a privacy checkbox.

Control over privacy, for Jordan, is much more than viewing your data. It is an individual decision. Some people want no privacy; others want a great deal. It is not a zero-one legal fact about which fields are available. A couple of hundred years ago most people lived in villages where everybody knew everything. That was not simply bad. You lost a kind of privacy and gained a kind of help: if something went wrong, someone knew and came. In a big city nobody knows and nobody helps. The answer depends on which end of that trade you are standing on. He wants to manage relationships — who knows what about him, and why — with real agency, not by drowning in unread policies and checkboxes.

He is not a privacy researcher, and he wants that admitted. The problem is not just technology, and it is not just legal scholars sitting with engineers. There have to be layers around it, the way electrical engineering did not stop at stuffing a plug into a wall. Underwriters Laboratories had to exist so the plug would not burn the house down. There had to be people who could install, and people who watched the installers, an onion of roles. Privacy is that kind of electricity. It could take decades to work out, and it will require structures we do not have. Those structures, if they are recognized as useful parts of ordinary life, are also where a lot of money will be made.

On the social layer he wants less anonymity and more locality: worlds you enter because you trust the people there, so a discussion is neither an attack nor a waste of time. He pulled out of Facebook because it was going to waste hours even when there was some value. He thinks human beings are fundamentally good, but limited — blinkered, slow to see another person's pain. On good days he thinks technology could open those blinkers; in his lifetime he has not seen it do so yet. Comment sections showed him a darkness he had not expected, and the stack now amplifies it. The progress he wants is modest: make it a little easier to be the decent person you already are.

Fridman asks whether a life or a society can be modeled as an optimization problem. Not in the way people usually mean, Jordan says. Optimization is one branch of mathematics, far too limited for that complexity. Even in this era, sampling sits beside it: optimization hunts for a single best point on a surface, sampling treats the same surface as a density and wants the high-mass regions. Lift the problem into the space of probability measures and the two can be made to look like the same thing, but mathematics that abstract starts to lose the theorems worth proving. As a statistician he takes the stochastic view without struggle. He does not know what will happen in the next five minutes. The best he can do is a rough sense of probabilities, then reason about what to do.

In multi-agent systems the game-theoretic answers appear. A particle in a potential well is optimizing a Lagrangian without running an algorithm; the math is a description for the analyst. The same will be true of mixtures of humans, computers, and markets. Nash equilibrium is a special kind of saddle point, so classical and algorithmic game theory were literally in the business of finding saddles. Any economist, Jordan says, knows Nash is often a poor explanation of behavior. Other equilibria exist, including Stackelberg: you move first, I watch, you anticipated my watching, and we go back and forth with a first-mover structure. Those objects have topologies. How dynamics approach them or flee them is sometimes known and often not, especially once the system is stochastic, decentralized, and large. Young people who think dense flow has finished the subject are wrong. The problems are open, and they are about strategic data collection as much as about descent.

Deep learning's loss surface, from an optimizer's point of view, is fairly smooth, and overparameterization leaves many paths downhill to a reasonable point, so reaching an optimum is less hard than high dimension led people to expect. Why some optima are much better, and why we often find those, still needs explanation. He doubts we will be looking at the same surfaces in ten years. Layers of arithmetic with a bit of nonlinearity did not come from neuroscience, whatever some of the inventors were picturing; arithmetic circuits showed up in computer science and control theory too. The surprise is that the idea works at this scale, not that we are stuck with it because we understood a brain.

Gradients themselves still shock people who study them. If a step on the x-axis gains three units and a step on the y-axis gains seven, and you may only move a unit of distance, almost everyone puts the whole step on y. That argument is wrong. The gradient keeps a little x in the direction, and you need the math to see why. Deterministic descent can slam into ugly features of a surface, including a kink like the absolute-value function where the derivative dies. Stochasticity makes hitting that point unlikely, and in high dimensions those properties are appealing for reasons our intuition handles poorly. Empirically, stochastic gradient was found to work; theory followed. The deepest idea he will name is Nesterov acceleration: under a model where you may only move in the span of past gradients, smooth convex functions have a 1/k² lower bound, vanilla gradient is 1/k, and Nesterov's two-gradient combination sometimes walks uphill, hits the bound, and is still being written about.

What is statistics? Somewhere between math, science, and technology, Jordan says: a set of principles for inferences you have reason to believe, and for decisions you have reason to think will not be errors. That requires saying what an error is and what the probabilities mean. Formally the subject is a few hundred years old. Probability first explained gambling: given a roulette mechanism, what outcomes do you expect over a long run? Then someone turned the problem around. Given the outcomes, what can you say about the mechanism? For a while the field was called inverse probability. Laplace, working in Napoleon's government, needed a census of France, gathered data, analyzed it for policy, and the name that stuck was statistics — the study of data for the state.

By the time the subject was formalized in the 1930s, game theory and decision theory were growing next door. People in that world did not think of themselves as computer scientists or statisticians or economists; they were all of it. Von Neumann was developing game theory and thinking about decision theory. Wald and the econometricians turned decision theory into statistics. You do not just analyze data. You have a loss, a question, a probability model, and a risk if you decide badly. To this day, Jordan says, most advanced statistical curricula start from decision theory, then branch into Bayesian and frequentist schools. The whole apparatus is about decisions.

He likens the Bayesian-frequentist split to wave-particle duality. The two are physically different, sometimes agree in practice, and sometimes give quite different answers; you get used to living with both. A loss is a function of data X and parameter theta, neither known in advance. The frequentist averages over X, holds theta fixed, and gets risk: a guarantee about how a procedure behaves across data sets you have not seen, the stamp you want on software shipped to the world. The Bayesian averages over theta, conditions on the X in hand, and talks only about this data set, which is the right move when you are working with a scientist and a domain. Both needs are real, so they have to fight a little and also blend. Empirical Bayes starts from the Bayesian math, notices quantities you do not know in the real world, plugs in a natural estimate, and then proves that the plug-in is not a disaster.

James-Stein estimation is the surprising idea he almost will not try to explain on a podcast; Stephen Stigler wrote a paper arguing that it defeats the mind's first attempt to understand it. A more accessible favorite is false discovery rate. You are not running one test. You have a bag of them. Among the ones you announced as discoveries, you want the fraction that are false to be small. That is not accuracy, precision, recall, sensitivity, or specificity, which mostly ask, given the truth, what data you would see. FDR asks the other direction: given that you made a discovery, what is the probability of the hypothesis? Priors sneak in, classical frequentists flinch, and empirical Bayes writes the formulas anyway until some of those unknowns can be estimated from the world. The line runs from Robbins around 1960 through Bradley Efron, Benjamini and Hochberg in Israel, and John Storey's Bayesian reading. Jordan has spent years absorbing it. He finds it a healthy way to think.

Fridman asks whether defining intelligence is even a useful question, and whether we will one day have benchmarks for general intelligence to put in front of machines. Jordan says that is really a question for psychologists, and the ones he knows are humble. They might try to understand how a baby tells solid from liquid, or when a verb parts from a noun. Human ability to take a complicated environment, abstract it, communicate it, and interact around it is staggeringly rich. He does not think anyone is aiming at that in the near future. Cognitive work on reasoning quirks is useful without being a deep account of how we reason. The post he keeps pointing to is the one titled "Artificial Intelligence: The Revolution Hasn't Happened Yet."

Step back, he says, and intelligent systems are not only humans, animals, or plants. A market that brings food into a city every day is a system of decentralized decisions. From far enough away it looks like a collection of neurons, each making a local choice, none of them knowing the overall goal, while something happens in the aggregate. People eat. The system is robust from village to megacity, adaptive in rain or shine, self-healing without a central tuner, and thousands of years old. It is not perfect, and intelligence never is. Humans did not evolve to *be* markets; we participate in them. Neurons can be viewed as a market, and economic neuroscience is an interesting path. The point is larger. If you spent thousands of years becoming the world's best psychologist and wrote down a complete theory of human intelligence, you might never have discovered supply and demand, matching, or auctions. Those are real principles. They lead to a form of intelligence that is not human intelligence, and there are probably third and fourth kinds nobody is thinking about hard enough.

Markets are relevant to computer systems right now. Understanding human intelligence, he thinks, probably is not. If you want general intelligence in any deep sense, it has to be this broader thing, not a copy of a person. Asked whether civilization is a single organism whose intelligence is rising linearly or exponentially, he declines. That is science fiction, fun over a beer and the wrong public conversation for people his age when there are present dangers and real systems to build. The brain-in-a-vat story he loved as a student — replace every nerve with a wire, then a radio, wheel the body away, kill the transmitters one by one, and ask where the person went — is worth an eighteen-year-old's philosophy class. It is not, he says, what most of us should be broadcasting.

Fridman asks what undergraduates should do if they care about machine learning, markets, economics, and psychology. First, Jordan says, the door is open. Second, it is a journey. You do not enter because you are brilliant and have brilliant ideas. You apprentice yourself: spend a lot of time, work on hard things, pull back, talk to many people, the way you enter any creative community. Musicians and artists do not wake up as geniuses on day one. They practice basics, stay humble about where they are, and accept they will never be expert at everything. Luck picks which branch of the tree you take. You will take some branch. Graduate school, he still thinks, is one of the wonderful phenomena we have: four or five years, apprenticeship with an adviser, a group you belong to, enough time to start from nearly nothing, grow some expertise, and watch your own creativity surprise you.

The work is far more cooperative than outsiders imagine. People teach each other constantly and are more than willing to show what they do not know. Jordan is happy to reveal his ignorance at a table so someone else can teach him. Because it is cooperative it is also international; most people in the field see no barriers. The nationalism of the current era, he says, is at odds with how they actually work. This is a human endeavor, done together, for everybody. When young people ask what to study, he tells them to follow the heart, do a lot of math because math is good for everybody, and also do some poetry, some history, some language, throughout a life. You want to be a thinking person.

He learned French as a late teenager in Kansas, bored, pulling his parents' French books off the shelf and teaching himself by reading. The first time he heard it spoken he understood nothing and still had the eerie sense he had known it in a previous life. Travel made the point: on trains in France he could talk to older people who had lived whole lives, see himself in their shoes, and work on empathy. Italian he picked up at MIT because the building was full of Italian professors and he was a young, slightly bored faculty member; later he married an Italian, and the household speaks it. He thinks Italian is more beautiful than French or English. Every language lets you say things a bit differently, and that exploration is one of the great pleasures of a life. If he had millions of dollars and infinite time, the scientific problem he would actually work on is natural language done right — deep understanding, the core of AI as he sees it, and a place we are very far from. He came to that too late to make a career of it. Learning languages, he says, is the small way he trained that part of the brain anyway.