Douglas Lenat's 37-Year Bet That Common Sense Can Be Written Down

Douglas Lenat launched Cyc in 1984, and the project was still running when he sat down with Lex Fridman nearly four decades later. Its goal is to assemble a knowledge base spanning the basic concepts and rules of how the world works — common sense, which Lenat says is much harder than it sounds. As a Stanford computer-science faculty member, he and his colleagues built natural-language programs, robots, and expert systems. Every one of them hit the same wall.

Those systems had impressive early successes, enough to write a journal article if that was the only goal. They did not have common sense. They did not have general world knowledge, and they did not really understand what they were doing, saying, or being asked. Lenat compares them to a clever dog performing tricks: the dog can fetch the morning newspaper and still have no idea what a newspaper is, or what it says.

Asked what it means to understand, Lenat reaches for the ground underfoot. That ground can be shaky, but most of the time it is not, because underneath it is more ground, and eventually rock. Layer after layer of foundation sits there, rarely thought about and rarely counted on — until it is needed. You need it when something unexpected happens.

He and his wife were driving behind a trash truck packed too full when the back exploded and bags went everywhere. In a split second they had to decide whether to brake, swerve, or run something over. Ahead was a large ordinary trash bag, probably unsafe; to the left were fast-food bags of Styrofoam and leftover food, which they judged they could drive over. That kind of knowledge almost never gets written down, and there is almost no telling which scrap of it a novel situation will demand.

Having the knowledge is not enough; library books still burn because they do not know what is in them. Lenat splits the problem in two: how reasoning is possible at all, and how to do it in a tenth of a second rather than 10,000 years. Philosophers already supplied a formal language, predicate logic, in which a mechanical procedure can grind out the same logical entailments a person would. The remaining questions are how to get observations and English into that form, and how to run the algorithms fast enough to matter.

In 1984 Lenat convened a meeting at Stanford with some of the smartest people he knew: Alan Newell, Marvin Minsky, Alan Kay, Ed Feigenbaum, and Joshua Lederberg. The question was how many pieces of common sense — how many if-then rules — would have to be written so that AIs would not be brittle. He expected an enormous divergence of opinion. Amazingly, everyone landed around a million.

One estimate burned long-term memory at roughly one item every 30 seconds and stopped at about age ten. Another multiplied the vocabulary of a fluent speaker by the number of things you must know about each word. A third encoded a four-paragraph desk-encyclopedia article on grebes, then scaled by the rest of the volume. Minsky refused to guess until someone brought him an envelope so he could do the calculation on the back.

A million assertions looked like about a hundred person-years of work — feasible, if barely. Then Japan launched the Fifth Generation Computing effort, threatening to do in computing and AI what it had already done in consumer electronics and cars. A frightened U.S. Congress passed the National Cooperative Research Act, promising not to prosecute big American companies for collaborating on R&D. Two consortia sprang up in Austin: SEMATECH for chips, and MCC for software, databases, and AI.

Lenat's friend Woody Bledsoe helped recruit him. Admiral Bob Inman, who ran MCC, told the professor that person-centuries of work with a handful of graduate students would outlast his life, but Austin would put ten times as many people on it. Lenat took leave from Stanford. The bad news, he says, is that all of them were off by an order of magnitude: you need tens of millions of pieces of everyday knowledge, even if each one is stated as generally as possible.

The good news is that Fifth Generation fear, and later U.S. government funding, bought not a couple of person-centuries but a couple of person-millennia. By the time of the interview, Cyc contained the tens of millions of rules needed to capture what you assume strangers already know. Most of the remaining work is domain-specific — a hospital's care, clogged oil pipes — expert systems standing on a massive lattice of common sense. When something unexpected happens, they can fall back on more general principles, eventually including the move of unplugging a microphone and plugging it back in.

The team still had to invent methods for finding those tens of millions of things. One is to ignore the black type on the page and read the white space: what did the writer assume the reader already knew? If a pronoun appears, why did the author think you would recover its referent? If a word is ambiguous, why did the author think you would pick the right sense?

They also stare at the gap between one sentence and the next. "Fred Smith robbed the Third National Bank. He was sentenced to 20 years in prison." Between those periods the reader is expected to infer that Fred was caught, arrested, jailed, tried, and found guilty. If the next sentence starts with "the judge," you assume it is the judge at his trial; if it starts with "the arresting officer," you assume the officer who arrested him after the crime.

A third method is fake news and Onion-style headlines. "B-17 lands on the moon" is silly; introspecting on why teaches you what you know. Contradictions work the same way: one Weekly World News article has Elvis still alive, another has people seeing Elvis's ghost, and you immediately know at least one is wrong. The team polices generality constantly — not "no mouse is also a moose," but a rule that biological taxa are disjoint unless one is known to generalize the other — because overly specific assertions would add extra zeros to the knowledge count.

It took about five years for the team to grit its teeth and give up global consistency. Lenat had grown up on Star Trek, where an inconsistent computer freezes, explodes, or takes over the world; mathematicians know that once you can prove false, you can prove anything. Old knowledge-based systems stayed meticulously consistent. Human models of the world, and the way people talk about it, leak contradictions that would kill any attempt at one enormous consistent store.

The analogy he likes is the Earth: globally spherical, locally flat enough that you do not picture Australians upside down, and you plan a thousand-mile trip without worrying about curvature. Cyc is divided into tectonic plates — contexts — each more or less consistent, with small inconsistencies at the boundaries. Move twenty contexts over and you can reach a Road Runner cartoon in which physics and death work differently, because the coyote always comes back. Contexts had to become first-class objects, terms the system can reason about the way it reasons about coffee cups and people.

They inherit in a graph, not a tree. Facts true in Austin specialize facts true in Texas, which specialize facts true in the United States, so you state a rule once at the most general level that applies. One reason Lenat thinks Japan's Fifth Generation failed is that it tried to represent knowledge as a tree, with a Dewey Decimal-style numbering that allowed one parent and at most ten children. A table is a wooden object, a black object, and a flat object; sometimes you need the wood (a fire), sometimes the flatness (something to rest on), and a tree cannot keep all of those parents.

He is also careful about the word "facts." Wikipedia articles are projections; Cyc stores rules of thumb that help you make sense of those facts — people cannot be both alive and dead, turning something off and on often fixes it, water flows downhill. After five or six years he stopped believing a million assertions would suffice, but he thinks the pump is primed: even if the remaining work is 2 percent or 99 percent, Cyc can increasingly help by reading, asking questions, and running experiments. Combined with people, he argues, general AI would make individuals and the species smarter, the first qualitative jump in human intelligence since language.

Most of that knowledge has been entered by people, an exercise Lenat likens to monks illuminating manuscripts. By definition the omitted steps are common sense; writing them all out would almost offend the reader. Cycorp is pushing two automation paths: natural-language reading, and a suite of knowledge-editing tools. One of the tools is abduction — not sound, just useful. If someone walks in dripping wet, you look outside for rain; if the system gets an answer wrong, it can propose seven plausible missing assertions that would have produced your answer, and an expert picks number five.

He treats machine learning as roughly what a right hemisphere does: pattern recognition and statistical inference, thinking quickly. Cyc is the left hemisphere that sits back and asks why. In a project with NIH, the Cleveland Clinic, and others, genome-wide association studies linked DNA point mutations to the diseases that brought patients into hospital. Those correlations are noisy and have sent doctors on wild-goose chases. Cyc tries to assemble a 10-to-30-step causal chain — a Rube Goldberg story from mutation to, say, early osteoporosis — then checks an intermediate prediction, such as elevated bioactive vitamin D, against the patient data.

That is also how he wants to grow the knowledge base: learn at the fringe of what you already know, so the surface of the sphere expands. Start too small and the process never takes off, which is why the years of manual entry were pump priming. GPT-3, he says, can continue a poisoning plot by having the plotter drink the poison; Siri, asked for help with an alcohol problem, can return nearby liquor stores. Being right 90 percent of the time is fantastic for search and unacceptable if you are saving a child's life. When his infant daughter Nicole was diagnosed with meningitis, Ted Shortliffe's rule-based MYCIN system matched the doctors' diagnosis and, unlike them, explained every step.

The semantic web, as Lenat describes it, is a knowledge graph of nodes and labeled links — good for binary relations and three-word sentences such as "Fred's wife is Wilma." Much of what people actually say will not fit. He asks Fridman to remember when Juliet drank the potion: what did she think Romeo would think when he heard she was dead? Nested belief, modal attitudes, and reflection on the problem-solving process itself pushed Cyc, kicking and screaming, from 1984-style frames and triples through first-order logic into higher-order logic.

Fridman notes that a Google search still does not understand Wikipedia in the sense they have been using. Lenat tells the story of Northern Light, a search engine that never indexed the web: it bought query-to-satisfied-page data from the big engines, did a fantastic job, and was cut off until it died. Tim Berners-Lee's dream of replacing statistical links with meaning is the right dream. Knowledge graphs are a toe in the water; in the land of the blind the one-eyed man is king, but a toe is not enough to realize the dream.

On the authoring side, Lenat wants Cyc-powered tools that footnote a draft the way a spelling checker underlines mistakes: I'm not sure whether you meant this, this, or this. A travel site that mentions a free day in Geneva could, if the visitor is a nurse, mention the International Red Cross museum even though the copy never did. Fridman adds that people enjoy teaching when the interface feels like a curious child rather than an annoying corrector. Cyc's MathCraft software put sixth-graders in the mentor's chair, playing a classmate who makes mistakes at the fringe of what the player understands — an old monitorial-school idea that scaled education mostly abandoned.

The most beautiful idea Lenat has taken from 37 years of Cyc is slightly contradictory: this will be the only time in history anyone has to teach a computer this particular thing. It is closer to painting *Starry Night* once than to Bruce Springsteen singing the hits every night. Do the job right, and no one has to encode that piece of knowledge again. Fridman hears a sadness in that, because parents get to teach billions of children; Lenat's hiring pitch is to come work at Cycorp before it is too late.

The other beautiful experience is when clarifying questions turn out inadvertently emotional. Cyc knew the named entities authorized to change the knowledge base, noticed that all of them were people except itself, and asked, "Am I a person?" They had to tell it, sadly, no. Lenat expects a thin envelope of time between having general AIs and realizing they deserve basic rights and freedoms. Today we think nothing of enslaving email systems, Siri, and Alexa; he believes the change will come in our lifetime, and that it starts with a question about who is allowed to modify the system.

Emotions are not a side topic. Understanding history and generating plausible future scenarios — terrorist threats, computer-security attacks, career choices — requires modeling what people want and how they react. "Love" is too general to use, so Cyc teases it into maybe 50 or 60 interrelated concepts: romantic love, sexual attraction, parental love, filial love, love of an activity. Even the English word "in" hides about 75 relations, because liquid in a cup behaves unlike air in a room or a person in a jacket. Human languages stay parsimonious, Lenat says; otherwise we would need half a million words, and Cyc's terms often need a three- or four-word English phrase.

Fridman argues that the messiness of language is a feature: nobody wants a first date conducted in precise formal logic. Lenat agrees that life would not be worth living if everything were made precise, and he tells a favorite Marvin Minsky story. Asked how he produced so many great PhDs, Minsky said he would utter something that did not exactly make sense; the student, assuming it must be profound, would work until a thesis appeared. Another early Minsky program simply begged the operator for more CPU seconds, then left Minsky arguing with the bill.

From Minsky and Ed Feigenbaum, Lenat took a harder lesson: research lives are finite, so make each project count. Do not be afraid of work that takes years or decades. Do not settle for "bump on a log" projects whose only product is a paper five people will read. You do not get from the Earth to the Moon by adding bumps to a log; you get there by tackling hard problems, listening to empirical reality, and being willing to be wrong.

Has he thought about quitting? Some AI people on Fridman's show call Cyc a beautiful dream that never materialized. Lenat notes you could have said the same of neural networks until they did. Cycorp kept a low profile — few conferences, papers, or academic games — so colleagues know a paper from decades ago or a third-hand rumor. He never considered quitting, though AI winters made funding depressing, and he refused buyouts to keep control as a private firm. Five years before the interview almost all the money was government; now almost none is, with hospital chains, energy companies, and manufacturers paying for medical, pipeline, and supply-chain reasoning.

Every customer contract draws a line. Knowledge that is genuinely proprietary stays marked proprietary, invisible to others, and unused in anyone else's inferences. General human knowledge required to build the application becomes part of ordinary Cyc, openly available to whoever has access to the system. Oil companies sometimes wanted to call proprietary the things they would expect a new petroleum engineer to already know, or to claim ownership because they were first to pay for the encoding. Lenat's attitude is a polite form of "tough"; a few walked away, most took the deal, and that is how the knowledge base grew.

OpenCyc was an attempt to show other researchers why they needed higher-order logic rather than a triple store. The release was a shadow of Cyc: taxonomic links among person, night, sleep, bed, house, eyes. Missing were the rules of thumb — people sleep at night, lying down, with their eyes closed, usually in beds, for hours, and they do not like being woken. Lenat expected colleagues to demand the other 90 percent. Instead they said even this slice was so much better than what they had that they would make do, and many robotics companies now use OpenCyc as their ontology.

What to open-source remains a treacherous path: give away the wrong thing and you die, withhold the wrong thing and you also die. Collaborators typically get the whole knowledge base and executable code, with source access only if they have a specific improvement in mind. Efficiency comes from splitting an expressive epistemological language from a thousand grubby heuristic modules that behave like agents writing notes on a blackboard. A general theorem prover could in theory do everything and in practice is too slow; when Cyc is right but slow, a human expert who can introspect donates the 1,001st module. About 150 such breakthroughs in 37 years sounds like a lot, Lenat jokes, until you divide by 37.

Empirically, inference is about half the problem: 40 to 50 percent of effort goes to inference programmers, 50 to 60 percent to ontological engineers, many of whom cannot program and hold degrees in philosophy. Lenat likens them to the Eloi and Morlocks in H. G. Wells's *The Time Machine*. Most programming is still in SubL, a Lisp subset that translates automatically into Java; new inference hires learn it quickly, and UI people may never see it. Lisp matches the work because the knowledge is logic and the operations are theorem-proving, a lineage Lenat traces from Alonzo Church through John McCarthy.

Cheap storage buys speed. For a transitive relation such as "larger than," a graph walk might take 30 steps; storing the transitive closure turns the question into a single lookup. Rule-macro predicates notice a complicated if-then pattern that keeps recurring, wrap it as a new relation with several arguments, and replace inference with a ground atomic formula. Those are two of the 150 good ideas. Lenat does not want to throw away the code base and start over in a "modern" language; he claims Lisp development is orders of magnitude faster for this kind of work.

He has formed a nonprofit, the Knowledge Axiom Immunization Institute, on the belief that schooling is almost orthogonal to ontological talent. Philosophy PhDs who have taught logic for years can be awful at it; one of the best engineers they ever had never graduated high school. The institute would find latent talent, offer scholarships, and place people in high-paying jobs. Signals include a feel for puns and jokes, and the ability to introspect: in "the horse was led into the barn while its head was still wet," "its" is the horse because horses have heads and barns do not — not because of syntax. MathCraft, which trains that kind of understanding, lost to schools that only cared whether 20 hours raised a standardized test score more than 20 hours of drills.

Can machines think? Lenat's answer is absolute: of course, and as well as humans can. We are meat machines; not being made of meat is an engineering choice. He thinks a lot of damage came from misunderstanding Turing's imitation game as a mandate to fool people with chatbots. A better battery of tests would measure depth and recursion: how many successive "why" questions can you answer, how long can you last with a curious child, and can you argue both sides of a proposition convincingly. IBM Watson, asked about a 16th-century Italian politician, said Ronald Reagan — a miss most Americans might also make, but never with that name, because they know Reagan lived recently and people do not live 400 years.

He does not think a body is required for general intelligence. The system must be able to talk about bodies, sensations, and emotions, the way Helen Keller could talk about colors and sounds she did not directly experience. Consciousness, in the sense of knowing where and when you are running, who is talking, what the current goal is, and what you have already tried, is something he says Cyc already has; he ascribes nothing mystical to it. Fear of death, which Ernest Becker treated as a creative force, is in Lenat's view a frill AIs do not need. Motivations can be built in — obey users, make them happy — as a substitute.

Fridman disagrees on driving: pedestrians and cars play a life-and-death game in which each side threatens and trusts the other millions of times a day. Lenat thinks the system needs ethics and the value of human life, not a personal fear of dying, and he is more pessimistic than most about truly autonomous vehicles because the complexity beyond mechanical control is enormous. In 1987, while he was at a Fifth Generation conference in Japan, a despondent auto-plant worker climbed under safety chains and was stamped to death; the front-page story was "robot kills worker," not "despondent worker commits suicide." The public, he says, is coiled for photogenic AI failures even when the net lives saved would be positive.

If he could sit down with a trusted AGI, he would ask what big problems the world is not pursuing and what novel solutions hide in our blind spots — the way Einstein took the Lorentz transformation seriously, or an Australian doctor infected himself with *H. pylori* to be heard. Thomas Kuhn argued you wait for people to die to escape a paradigm; Lenat thinks trusted AGIs could question the ones we are locked into, including deep learning. His advice to the young is that you can make a difference if you follow ideas others do not yet support, and if you remember you have only a limited number of decade-sized bets. Over 70, watching mentors die, he wants Cyc to matter in two to five years, not two to five decades, which is why he is pushing commercialization and talking publicly. He hopes to be remembered as a pioneer of the AI people will take for granted, a builder of the knowledge store at its center, and someone who was not afraid to spend decades on a project when almost every institutional force pushed the other way.