A Bar in Montreal, Then a Night of Code: Ian Goodfellow on Why GANs Work

Lex Fridman opens with the Russian-doll diagram from Goodfellow’s textbook *Deep Learning*: deep learning sits inside representation learning, inside machine learning, inside AI. The picture implies that deep learning is not the whole of intelligence. Goodfellow’s first limit is data. The methods still want huge labeled sets; unsupervised and semi-supervised tricks cut labels but still need piles of unlabeled examples; reinforcement learning skips labels and instead burns through an absurd number of trials. “As human beings, we don’t learn to play Pong by failing at Pong two million times.”

A second limit is placement. Nobody serious, he says, is proposing deep learning as the entire ingredient of intelligence. AlphaGo uses a net to estimate a value function. Most reinforcement-learning algorithms use a net to estimate the next action. The rest of the system is assembled around those modules. Fridman asks whether nets could reason the way 1980s symbolic systems did, more like programs than like functions. Goodfellow already thinks of them as programs: depth is the number of steps that run in sequence on a flowchart or a TensorFlow graph; width is the number that run in parallel.

That is a different story from 2006, when Jeff Hinton showed deep belief networks could be trained and the excitement was layered abstraction: edges, then corners, then grandmother cells for specific objects. Today nobody believes layer 150 of a ResNet is a grandmother cell and layer 100 is a contour detector. Residual nets take one representation and update it many times. What arrived with deep learning, Goodfellow argues, was serial steps. A support vector machine can multiply thousands of features by weights, but those multiplications happen in parallel. The extra power is the multi-step program, not a new kind of representation at every floor.

He defines deep learning as any machine learning that learns the parameters of more than one consecutive step. Each step refines the same representation rather than replacing it. In some sense that is like reasoning: not deduction, but taking a thought and polishing it until it is good enough to use. The line between deep and shallow, on this view, is whether anything runs in series at all.

Fridman asks whether human cognition or consciousness could simply be sequential representation learning. Goodfellow splits the word. Self-awareness — an agent that knows it exists in the world and plans on that fact — is something a computer scientist can turn into an assignment. Reinforcement-learning algorithms are already forced to model their own effect on the environment. Trained well, they already have limited versions of that kind of consciousness. The jump from limited to human-level, including common-sense reasoning, is the open question.

The other sense of consciousness is qualitative experience, qualia, the philosophical zombie who does all the same information processing with nothing it is like to be that zombie. Goodfellow has no idea how to make that a scientific question. He does not know an experiment that would tell you whether a person is a zombie, and he does not know one that would tell you whether an advanced AI system had “woken up” in that sense. Cognition, he is willing to say, can emerge from current-style architectures. Qualia he leaves off the lab notebook.

On scaling, he is optimistic about more compute and more data, but only if the data is the right kind. Almost every model today is trained on one type of data. A human life stacks vision, hearing, touch, riding a bike, driving, talking, and reading in the same lifespan. His bet is that an integrated dataset, used by a model that can close the loop and interact, may not need algorithms very different from today’s — scaled up on a lot of multimodal data, they may learn something genuinely interesting.

Fridman turns to adversarial examples, Goodfellow’s other long project. Early on he treated them as proof of a big gap between machines and people, something to close. After more study he still thinks they matter, but he now reads them as a security liability rather than as evidence that machine learning is uniquely broken in a way humans are not. Used as a training tool they are less magical than they first looked. In 2014 his first adversarially trained classifier, on MNIST, resisted some attacks *and* got more accurate on clean digits. That result has been replicated: weak adversaries can make MNIST classifiers better.

The bonus does not travel. On other datasets, and against stronger attackers, you tend to give something up. Accuracy on clean examples and robustness to a real adversary trade off. The idea is still compelling because it matches how people learn — think of how you would fail, then fix that — and because worst-case analysis is ordinary engineering. If a system works in the worst case, it usually works in the messy average case of the real world. Autonomous driving already hunts for ways to break the stack; surviving those cases is a hand-waving empirical argument that the system is trustworthy.

The stakes are concrete. A trading algorithm that always makes the same mistake on the same input is a gift to someone who wants to force bad trades. Speech is worse. A 2016 paper, *Hidden Voice Commands*, produced sounds a person could not understand that a phone heard as the attacker’s phrase. Since then the attackers have moved faster than the defenders. It is now possible to make audio that a human hears as ordinary speech, with a little rustling in the background, while the phone hears a completely different sentence. The rustle is the perturbation.

Peter Norvig asked Goodfellow to write the deep-learning chapter for the fourth edition of *Artificial Intelligence: A Modern Approach*. He waited about a year before putting words down. Having already written a full textbook with Yoshua Bengio and Aaron Courville, he could see which topics had stood a few years of publication and which now looked extraneous. The field had also stabilized enough that some 1980s core ideas were in use again. When he first studied machine learning, almost everything from the 1980s had been rejected; the pieces that came back are what he put in the chapter.

There are two philosophies of a book: a reference that covers everything, and a high-level summary that gives readers the language of a field and names the important concepts. The first *Deep Learning* book tried to be both. For Russell and Norvig he chose the concise introduction. On a fast-moving topic such as learning-to-learn, he wrote a paragraph that said: this exists, it may be where your next convolutional module comes from, and there is no point cataloguing which architecture hit which score last month.

Pressed for a classroom definition, he repeats the sequential-steps test. Shallow learning can involve many parallel operations, even hand-designed feature extractors, but only one step is learned. Deep learning is multiple parameterized operations in sequence: the popular convolutional and recurrent nets, and also things that have died out, such as Boltzmann machines that did not use backpropagation. Many people now use “deep learning” to mean gradient descent on differentiable functions. He calls that a legitimate usage, just not his. A convolutional net trained by evolution would still count. Boltzmann machines, where you cannot take a useful derivative, still count if inference applies many parameterized steps.

Hinton has said we should throw backpropagation away and start over. Goodfellow’s reply is closer to engineering history than to revolution. When an algorithm leaves the critical path of research, it does not die; it becomes specialized. Logistic regression does not excite people working on speech recognition or autonomous cars. It still has plenty of use analyzing noisy medical and financial data, or making very fast predictions under a hard time limit. Backpropagation and gradient descent are allowed to stay. They may not be everything needed for human-level or superhuman AI.

He can imagine stacks of models in which lower models predict the parameters of higher ones, so the top level is predicting how different values will perform rather than computing gradients. Bayesian optimization already does a version of this: a Gaussian process predicts how hyperparameters will do. Plenty of non-backprop methods work well on specific problems. What nobody has done is take one of those methods and actually advance the state of the art on an AI-scale task. He would not be surprised if an existing algorithm, customized, eventually did something interesting at the level of cognition.

The system he says we do not have working yet is short-term memory. LSTMs are named for long short-term memory and still do not do what a person does. Gradient descent, to store a specific fact, has to take many steps on that fact. Tell a person the meeting is at 3 p.m. once and they have it; you do not repeat “three p.m.” until the weights move. Attention and Neural Turing Machines that write to memory cells have tried to update on a fact immediately. He does not think the field has nailed it. New optimization, or a new use of old optimization, that can lightning-fast write a single fact into state is still missing.

Symbolic systems of the 1980s assembled facts better — graph search, first-order logic, entailment — at the cost of heavy expert input and a narrow scope. Goodfellow’s own work, mostly ML security and generative models, has not gone back there. He can see a middle path: a differentiable knowledge base, or an interface that lets fuzzy learners talk to a knowledge base. A neural net is already a kind of differentiable knowledge base. Generated faces are often asymmetric, with mismatched eye colors far more often than in life. A store of hints such as “faces are roughly symmetric” and “eye color usually matches,” injected instead of rediscovered from data, would help. Extending 1980s tools so they play nicely with nets is one way there; better language is another.

Fridman asks for the bar story. In 2014, in Montreal, Goodfellow argued with people from the lab that two neural nets could train as opponents. Almost everyone objected. He does not want to promote drinking for science, but he thinks lowered inhibitions helped: he is less likely to shoot down his own ideas after a drink. At lunch he would probably have agreed that you cannot train a second net in the inner loop of an outer one. That was the friends’ objection. Training two nets at once would be too hard; they would lose synchronization.

The worry came from experience. The lab, including Goodfellow, was deep into Boltzmann machines. Those models run two processes at once: a positive phase that loads data and makes it more likely, and a negative phase that draws samples and makes those samples less likely. Sampling itself is an iterative inner process. On MNIST the two processes worked. Color photos broke them; nobody in the lab, Goodfellow included, could keep the phases aligned. Friends expected the GAN discriminator to have the same problem as that negative phase: it would never keep up with the generator in the outer loop, and nothing useful would converge.

He went home around midnight, coded it, and it ran. Five years later he still cannot put a finger on one factor that explains why GANs generate photos and deep Boltzmann machines do not. Theory papers show the algorithm converges in restricted settings; those settings are too tight to explain everything seen in practice. His modest conclusion is the one many experimentalists reach: a lot of machine-learning algorithms cannot be predicted in advance. You run the experiment.

A generative model trains on a set of data — photos of cats — and then either produces more cats or estimates a distribution so you can ask how likely a new image is to be a cat. Some models are good at samples, some at density. GANs lean toward samples. A few variants, such as Glow, can do both. Sample generation is from scratch: not compositing one cat’s eye onto another’s ear, but a digestive process that forms a representation of the distribution and then invents a cat that never existed. The special structure is a two-player game. The generator starts at noise. The discriminator is trained to say “real” on training photos and “fake” on generator output. Both get better. Game theory says there is a Nash equilibrium where the generator has the right distribution and the discriminator cannot do better than chance, because real and fake look equally likely.

Fridman asks whether Goodfellow ever sits back, slightly stunned that this works. Goodfellow turns the amazement around. Every generative model has the property that if it really did what we asked, it would do nothing but memorize the training data. Maximum-likelihood models put all their probability mass on the training examples. The GAN game is played on a training set; the unbeatable strategy is to memorize those examples. A former intern wrote a statistical-learning-theory paper arguing that memorizing is actually hard: it would take many learning steps and many observations of many variables. That still does not explain the other half. Why, when the samples are new, are they convincing pictures rather than garbage?

The gap between possible images and the images a model sees in training is enormous. Part of an answer is a paper called Deep Image Prior: a convolutional net whose parameters are never learned is already useful for inpainting and denoising. The architecture itself encodes something important about image structure. A sizable share of the progress we see in image generation may come from that architecture rather than from learning. The implication is less cheerful for other domains. Speech models can be made to look reasonable, but biology datasets — microarrays of enzyme levels, for example — are rarely modeled this way. Vision’s success partly reverse-engineered the human visual system. That trick may not copy to arbitrary data.

Most other generative models are likelihood-based: maximize the probability of the training points. For rich images or audio it is often computationally impossible to evaluate the density of a model you would write down intuitively. One school designs models that stay tractable, including autoregressive models such as PixelCNN, which factor the distribution into a product over pixels. Density can be computed roughly in parallel; generating still tends to go one pixel at a time and can be slow. Image quality is reasonable. Goodfellow says many of the best samples now come from GANs, but it is hard to separate the algorithm from the people: graphics and art researchers flocked to GANs, and GANs also prioritize sample realism over an accurate density.

The 2014 paper showed only that the thing ran. MNIST digits and small grayscale faces from the Toronto Face Database were recognizable — colleague Bing Xu built the first GAN face model for that paper. CIFAR-10’s 32×32 cars, cats, and dogs were not. Deep-learning researchers then were used to staring at failed samples the way some people read tea leaves. The leaves looked different, not necessarily better, and that was enough to get people excited. The next real step was LAPGAN, from Emily Denton and Soumith Chintala at Facebook AI Research: a pipeline that generated from low resolution up to high resolution and, for the first time, produced genuinely good high-resolution GAN photos.

Later that year Alec Radford, Luke Metz, and Soumith Chintala published DCGAN. The name is no longer unique — almost every GAN is deep and convolutional — but the paper was a recipe: one model, no multi-scale pipeline, realistic faces. Goodfellow’s analogy is the backbone in animals. Once you have a spine, fish and four-legged animals arrive in a hurry. DCGAN became the skeleton for a crowd of later models and is still used as a baseline. That was the start of the Cambrian explosion.

The more interesting later work, he says, is not sharper pictures but new uses. The discriminator can be turned into a classifier: not just real or fake, but cat, dog, car, truck. Tim Salimans’s paper *Improved Techniques for Training GANs* — Goodfellow is a co-author and says this part is not his credit — showed you can train that classifier with far fewer labels. On MNIST, getting under 1 percent error had taken about 60,000 labeled digits. The semi-supervised GAN did it with 100 labeled examples, a roughly 600-fold drop. It still used more images than that; it just did not need a label on each one. Generating a recognizable object of a named class still needs some labels, because the model has to know what “cat” means.

A later semi-supervised GAN paper from Google Brain, built on BigGAN, matched BigGAN’s ImageNet results with about 10 percent of the labels. BigGAN had trained on 1.2 million fully labeled images. The method is close to clustering: the discriminator groups objects, and every generated image has to come from one of those archetypal groups rather than from an arbitrary pile of pixels. With no class labels at all you tend to get grass, water, brick, dirt, and not much of a subject. On a large ImageNet photo the object often does not fill the frame, so the model learns realistic pixels without learning that an object should be the star of every picture. CycleGAN’s horse-to-zebra map makes the same point. Horses usually stand on grass; zebras usually stand on drier ground. Convert a horse and the model often dries out the background. It learned the joint, not how to cut “zebra” away from “dry terrain.”

Goodfellow spends most of his adversarial time on security: an attacker trying to break a system, a defender trying to make it resilient. Domain-adversarial learning is the cousin that looks most like a GAN. The authors had the idea before the GAN paper and later cited it. The practical problem is ordinary. You train on a clean set such as ImageNet and deploy on phones: dark photos, motion blur, bad composition. A normal model often collapses because the new inputs look nothing like training. Domain-adversarial training learns a feature extractor whose features have the same statistics in both domains. The other player is a domain recognizer that looks at those features and guesses which world they came from, like a real/fake discriminator. The extractor has two jobs: fool that recognizer and still produce features that classify well. Sometimes both domains get worse so they match. Often both actually improve.

The naive hope for GANs as data augmentation — train a GAN, dump its samples into a classifier — is something he has never heard work. He sees no reason a GAN should generalize *better* than the classifier trained on the same set. It might generalize *differently*. A smarter experiment, which he has not run, would be to train many generative models on one training set, sample from all of them, and train the classifier on the mix. Each generator might capture a different axis of variation, a kind of ensemble of generators rather than of classifiers.

The more distinctive use is generating data with *different* properties. Casey Greene’s lab showed a GAN trained with differential privacy: samples inherit the same privacy guarantee as the parameters, so you can release fake patient records other researchers can analyze, with a quantitative bound on how much any real person’s data leaked. Fairness has a similar trap. Dropping the gender column is not enough; gender is easy to recover from a name. What you want is a model that still uses many attributes, makes an informed prediction, and is not reverse-engineering the sensitive variable inside. The recipe is almost domain-adversarial: a feature extractor versus a feature analyzer that must fail to guess the protected attribute. Goodfellow calls it quite brilliant and simple.

CycleGAN suggests a further audit. Horses become zebras; day photos become night. In principle you could take records from one group, map them to analogous people in another, and test whether the system treats the two versions equitably. He is careful. There are too many ways for the conversion itself to be unfair, and he does not think this is close to something you could actually use. Designed very carefully, it might still be an audit: move this person from this group to that one, and see whether the system still treats them as it ought to.

Asked about malicious identity fakes, Goodfellow is much less worried about twenty years from now than about the next few. There will be a bumpy cultural transition while people meet, for the first time, video and audio that look real and are not. Twenty years out, people will mostly understand that seeing a video is not a reason to believe it. They will expect cryptographic signatures or some other mechanism that argues the content is genuine. A startup called Truepic already sells ways to authenticate that an image is real. He does not claim those defenses would survive a determined state actor, but people are working on it, and he thinks the path eventually works.

He does not believe we will ever look at pixels and know for sure. Relying on that approach is dangerous. Build a very good fake detector, then someone fools it, and the detector says the image is not fake: the picture is now *more* credible than if no detector had existed. The detector becomes a false endorsement. Behind the scenes, estimate systems can still help, perhaps not as courtroom proof. Stronger authentication is the better bet. Imagine every phone hardware-embeds a private key and signs everything that comes out of it. You still could not prove the scene was real, but you could prove that someone who knew that phone’s key signed this image onto this server at this timestamp. A state agency might plant a key or crack a chip. An adversary with fewer resources would have a much harder time faking things for most of us.

Are there still ideas you can check as fast as GANs? Plenty, he says, but the one-hour timescale was an outlier. Low-resource ideas that do something very different at the algorithm level and get a big payoff are less likely now in core technology: a better classifier, a better RL algorithm, a better generative model. If he had the GAN idea today he would have to show it on ImageNet or a high-resolution set, which take a long time to train. You could not train for an hour and know you had something new. In 2014, MNIST was enough.

Other corners of machine learning still look like that everyone-is-an-amateur stage. Fairness and interpretability, he says, we do not even know how to do. On interpretability we do not have the right definitions. Defining a genuinely useful concept, with no experiment at all, could move the field. Differential privacy is the analogy. Before Cynthia Dwork and collaborators wrote a technical definition, privacy was mushy. Afterward you could design randomized database algorithms and prove, quantitatively, that each person’s privacy was preserved. Once the definition existed, algorithms arrived quickly.

Today everyone talks about how interpretable their method is, and everyone has a different picture in their head. A measurable quantity related to interpretability would be a huge leap even without a new algorithm that increased it. After a definition of differential privacy, the algorithms that guaranteed it showed up fast. The same could happen here.

Human-level intelligence, Goodfellow says, will take better training environments, a wide diversity of experience, and a lot of computation. He does not think AGI arrives by training on fixed datasets or by sitting still and thinking hard. The agent has to interact, and it has to collect a variety of experiences inside the same lifespan. Today we have many models that each do one thing, each trained on one dataset or one RL environment. Some papers get one set of parameters to do well in several environments. We do not have an agent that moves seamlessly from one kind of experience to another and integrates what it did over a life. Multi-agent setups are usually homogeneous: a pile of similar action video games. He does not see an agent that goes from a game to the *Wall Street Journal* to predicting whether a molecule will work as a drug.

His intelligence test is more engineering than Turing. Instead of downloading CIFAR-10 and writing a Python parser, you point an agent at the CIFAR-10 problem and it downloads, extracts, trains, and starts giving predictions. A system that does not need every step of the pipeline assembled for it understands what it is doing. He draws a line against AutoML, which mostly asks whether, once humans have built all the glue, the learner can design the architecture. If something knows how to preprocess the data so that it actually finishes the task, it becomes hard to argue that it does not understand the task in a fundamental sense. He is not sure that is the philosophical definition of intelligence. It is something he would love to build: give it a Wikipedia URL, or a paragraph of what you want, and let it decide what to search and download.

Fridman notes that both of them speak with so few filler words that commenters have called them robots, and asks Goodfellow to prove he is human. With today’s technology that is easy, Goodfellow says: this conversation has not veered into the stock market because of his training data. In general, proving something is real from content alone is incredibly hard. That is one of the main things he has taken from his research. You can simulate almost anything, so you have to step back to a separate channel. He jokes that he should have put himself on a blockchain at birth. He did not, so he cannot use his own methodology.

The near-term problem he is most excited to see solved is resistance to adversarial examples: making machine learning secure against someone who wants to interfere and control it, in images, language, and driving. He is more worried about domains that do not exist yet. Ask people in 2002 for the important problems in phone security and they would not have pictured what phones do now. AI will be like that; the business opportunities that take off are hard to predict. What you can predict is that almost anything you do with machine learning, you will want to stop someone from hijacking with a funny QR code or a funny input pattern. One category of solution he likes is models that change every time they predict. A frozen rule that always maps the same input to the same output is a sitting duck: find a profitable mistake once and reuse it. A model that updates is harder to steer. It keeps a little mystery because it keeps changing. Content can be perfectly faked. Credibility has to come from a channel outside the content — for a video, a face, or a model that always returns the same answer.