The Algorithm Is Already Better. The Clinic Still Uses 1967.

Regina Barzilay tells Lex Fridman that she spends so much of her life inside MIT that a lot of what she knows about the world comes from books rather than from a wide circle of people. Asked for something outside computer science that changed her, she names *The Emperor of All Maladies*, a history of how cancer treatments and drugs were developed. Even though she is already "in the business of science," the book opened her eyes to how imprecise the discovery process is, how imperfect current solutions remain, and what actually makes an idea succeed and get used. Sometimes, she says, "it's actually not the strengths of the idea, but devotion of the person who wants to see it implemented."

That devotion, she adds, matters locally even if it may not matter asymptotically. In natural language processing, statistical methods took a long time to become mainstream in part because prominent personalities did not believe in them and slowed the work. Fridman steers her back to the cancer book, and she describes a lineage most computer scientists never see: nineteenth-century dye chemistry in Germany and Britain, researchers noticing that certain molecules did things to cells, and from there a path into drug development.

The chemotherapy chapters are less inspiring. Some of the drugs were developed in Boston, including work associated with Dr. Farber at Dana-Farber, and the experiments involved "some miscalculation, let's put it this way." They tried treatments on children with leukemia; the children died; then the team tried another modification. Looking sixty or seventy years back, she can understand that mess. What shocked her were stories in the same book that happened only decades ago. "We still don't have a vehicle," she says, to make the process "much more fast and effective."

Fridman asks, from an engineer's vantage, how far we are from understanding the body well enough to manipulate it and cure disease. Barzilay turns the question around. One reason computer science succeeded in the domains where it did, she says, is that it is not really trying to understand. Amazon does not understand you when it recommends a book; it matches patterns in your behavior. Traditional marketing sliced people into subgroups, named their features, and designed a strategy for each slice. Recommenders make no such claim.

Traditional biology, as she encounters it, puts enormous weight on mechanistic understanding — more than she expected coming from computer science. Given the complexity of the system, a deterministic, complete account of these processes may simply be beyond human capacity. Face recognition and recommendation are probabilistic matching. In some cases, she suggests, "maybe we shouldn't even attempt to understand," or we can attempt to understand while, in parallel, doing the matching that finds a cure or an earlier diagnosis.

She knows that in biological journals and medical communities, "understand" is an important word. She still wants to know what it is supposed to mean here. Fridman offers the old symbolic-AI dream: compress human knowledge into logic and play with that logic in a way people can explain. Barzilay understands why that dream is beautiful. What seems to work today, they agree, is reducing the problem you care about to data and applying statistical methods.

Barzilay was diagnosed with breast cancer in 2014. Fridman asks how facing mortality changed her; she says she has been interviewed many times and nobody had asked. She was forty-three. It was "the first time I realized in my life that I may die," and she had never thought about it before. For about two and a half months she did not know where she stood. Tests arrived one after another; one would look bad, the next would not. Until she knew her actual position, she ran through every possible outcome.

She does not remember a clean split between optimism and dread — it was a mixture, in her phrase, "with many components." Each result brought a spike of hope, then of despair. After treatment she came back to MIT with her brain "not really there," still teaching. She looked at what her group was doing, and at what other groups were doing, and saw trivialities: people building careers on two- or three-percent improvements. She had been working on deciphering a eukaryotic "language that nobody speak." Walking out of MIT, she met people who asked about her next paper — and she saw suffering she had been "totally shielded" from on a daily basis.

The split was practical, not only emotional. Her graduate students wanted papers and wanted to continue work that was already going well; she had just re-evaluated what counted as important. Looking back, she remembered topic models as the hottest thing a decade earlier, with "millions of papers" and variations, most of it now irrelevant. What you perceive as important fades. Because time is limited, she argues, you have to take time to understand what matters to you — which may not be what the rest of the scientific community thinks matters — and then pursue that vision.

Radiation meant going to the hospital every day. She saw a community of people, many much sicker than she was at the time, who could be happy simply because they felt a bit better that day. In ordinary life, feeling well is taken for granted: you decide to go running, and you run; you are more or less free to do what you want with your body. Her community became those patients.

A friend, Dina Katabi, took her to Boston's Prudential mall to buy a birthday gift — the first time in months she had gone out among other people. The shoppers were happy and laughing, and to her they looked "totally crazy," wasting money on "stupid gifts." They might die; they might already have cancer and not know. Of course she had always "known" she would die. It was a theoretical notion. Once lethality was concrete, she felt that "we need to take a lot of our brilliance that we have here" and translate it into something useful.

Fridman puts numbers on the mission: on the order of 1.7 million new cancer cases a year in the United States, and about 600,000 cancer-related deaths. Alleviating that suffering, he says, is a beautiful use of a research life. Barzilay will not give a date when civilization will "cure cancer." She cannot assess that. What she believes machine learning will do is let many types of cancer be predicted much earlier, and let existing treatments be used more effectively.

She also hopes that AI plus drug discovery will find relevant molecules much faster than the current pipeline. What she is not sure about is how long the medical establishment and regulatory bodies will take to catch up and actually implement any of this. That delay, she says, is a large piece of the puzzle that is currently not addressed.

Early detection matters because, for many cancers, finding the disease sooner sharply changes outcomes. The brutal example is pancreatic cancer: the vast majority of patients are detected at an incurable stage, which is why five-year survival is only a few percent — "pretty much a death sentence." If the disease is found early, there are mechanisms to treat it. She knows people who were saved because they went to the emergency room with food poisoning, a scan showed early signs, and that accident saved their lives.

As the tools get better, many more people who are likely to develop disease could be seen in time instead of found by chance. Going deeper into the field also taught her that cancer is not the only horror on the list. Neurodegenerative and aging-related diseases still lack good solutions; we fixate on cancer partly because it is so prevalent. Computer scientists, she says, have quietly assumed that treating those diseases is someone else's job — biologists, chemists, physicians. That assumption is wrong. "We all need to join the battle."

For many cancers, we still do not know a given person's likelihood of getting the disease. For the vast majority of patients, especially younger ones, diagnosis comes as a surprise. In breast cancer, "80 percent of the patients are first in their families." Barzilay was one of them. She never thought she was at increased risk, because nobody in her family had had it; in her head it was an inherited disease.

Even the simplistic statistical models used in clinical practice would not have given her a real answer. The same blindness holds for pancreatic cancer, for lung cancer in non-smokers, and for many other diseases. Machine learning's job, as she frames it, is to use the data already sitting there — imaging, other tests, and eventually liquid biopsies — to say who is likely to be susceptible. A single signal may be too weak for the human eye. A model trained on large volumes, combining sources, can detect earlier. That pattern, she says, is already visible in breast cancer, and other groups are reporting it in other diseases.

All of this boils down to data, and data is the ugly part of the campaign. After she decided she wanted to work on the problem, it took two years to get access. In the United States, she says, there is no publicly available dataset of modern mammograms that a researcher can simply sign for and download. It does not exist. Hospitals have their own collections, and some data have come out of clinical trials, but a computer scientist who just wants to run a model on imaging is out of luck.

The one public resource she mentions is a film-mammogram dataset from 1993, which is not representative of current machines; whatever you learn on it does not scale. Hospitals hold the data and decide whether a researcher gets it. There is a proper IRB-guided approval process, and if you go through every step you can eventually get access. In practice, few people in the community have good access, because the process is so hard.

The images are already stored digitally, in the right format; no extra scanning step is required. Technology is not the bottleneck. Legal liability is. The hospital is legally responsible for the data. It has a lot to lose if it gives the data to the wrong person, and as a legal entity it may not have much to gain if it gives the data to the right one.

Her concrete proposal sounds like the organ-donor checkbox on a driver's license. Whenever someone goes to the hospital, it should be easy to donate data for research, at a chosen granularity — test results only, mammograms only, or the whole record. "We all will benefit from all this insights." The trap is familiar: people want to keep their own data private while hoping everyone else's is used. If patients could set those permissions themselves, she says, it would be a game changer.

Privacy fears are real, and trust has to be solved before large datasets can help the same people who worry. Barzilay splits the answer into technical and societal solutions. On imaging, de-identification — removing faces and names — already works fairly well. Free text cannot reliably hit 99.9 percent. A more interesting technical path, some of it developed at MIT, is to learn on encoded data: encode the image locally, train a network only on the encoding, send the outcome back to the hospital, and decode there. Learning happens in encoded form. The work is still early, but she thinks the area will move.

Even today, some data can be shared cleanly — lab numbers with the name stripped, for example. The bigger problem is still that hospitals see little incentive to give data away. She remembers needing an answer, having to make a choice, and having no information to make it: you feel that "your life is at stake, but you just don't have information." After talks, women email her in that same position. Almost every week a mammogram arrives by mail at her office, from people who need a life-changing decision. She is not opening a clinic; they run the statistics and send results to the patients' doctors.

"We all at some point or our loved ones will be in the situation where you need the information to make the best choice." If that information does not exist, you feel vulnerable and unprotected. Fridman asks whether patients could walk out with a USB stick, or upload records the way Microsoft and Google Health once promised. She thinks patient-controlled exchange is the right direction: today, sending a mammogram still means finding a small hospital office and using a decades-old mechanism. He notes that both consumer health-cloud efforts closed. She cannot fully explain why healthcare IT so rarely succeeds, and she finds that implementation failure almost as interesting, anthropologically, as the models.

Even when researchers have data, it is often unclear what evidence would actually change the standard of care. Traditional breast-cancer risk assessment uses "density": roughly, how much white you see on a mammogram, taken as a sign of dense tissue. The idea dates to 1967, when a radiologist named Wolfe looked back at women who had been diagnosed and asked what image features might have predicted that. He coded what the human eye could see into four categories. Those categories are still in use.

In 2019 that density assessment became federal law, approved by President Trump and pushed by the previous FDA commissioner: providers must advise women if high density puts them in a high-risk category. In some states, insurance will pay for supplementary screening on that basis. How much biological or epidemiological science sits behind it? Between 40 and 50 percent of women have dense breasts, so about 40 percent of patients leave screening told they are high risk. If you label half the population high risk, she asks, maybe she is not; and what is she supposed to do, given that the system cannot offer expensive follow-up to that many people?

Density became a big deal because patients pushed. Many women had "normal" mammograms and later turned out to have quite developed cancer; they had no way to know who was really at risk, or what "you're OK" from a doctor actually meant. Fifteen years earlier, density may have been the best science available, and it took fifteen or sixteen years to become federal law. Now it is the standard — and a deep-learning model can predict who will develop breast cancer much more accurately, because it is trained to look at the image and forecast one-to-five-year risk. The machine systematically finds the patterns Wolfe's eye was reaching for.

How long to substitute density? "It's not a matter of the algorithm. Algorithms [are] already orders of magnitude better." The live questions are who you need to convince, how many hospitals need to run the experiment, how adoption works, and how you explain to women across the country that the new measure is actually better. Making the model still better will not solve that. The book she is reading, Elisabeth Rosenthal's *American Sickness*, came from her clinical collaborator Dr. Connie Lehman; Barzilay thought she already knew the American health system, and every page still surprises her. For computer scientists entering medicine, the lesson is the incentive structure, and the need to show concrete changes in care rather than hype about having "killed" cancer.

Detection is only half of the job: you still need drugs. As of this 2019 conversation, Barzilay says there is no drug developed by a machine-learning model, and none in which a model even played a significant role. Generating molecules with desired properties, and using them in screening, is in her view a major open problem. Diagnostics and imaging, she admits, mostly take ideas developed elsewhere and tweak them. Drug design is different. It is technically interesting.

Small molecules — still the majority of drugs — are graphs: atoms as nodes, bonds as edges. You can work in two dimensions or three. Industry, without machine learning, uses high-throughput screening: you want a biological activity, you physically test hundreds of thousands of compounds in the lab, and you keep the hits. Then chemists, using domain knowledge, modify the starting molecule for solubility, lower toxicity, and fewer side effects. Their creativity determines success, and that is how currently FDA-approved drugs are designed.

Measured by cost, the process is prohibitive. Measured by time, many diseases still have no drug at all, and neurodegenerative programs in particular are littered with late-stage failures that are financially catastrophic. Is it the most effective mechanism? "Absolutely no." It is the only one that currently works. She has spent time with people in pharma and is impressed by how sharp they are and how deep their observational science runs. She still wants the process changed, because even the most experienced chemist cannot hold in memory what can be learned from millions of molecules and reactions.

The first tool her group built predicts properties of a given molecule. Early work used handcrafted fingerprints — breaking the graph into substructures — and a feed-forward network; that was not enough. You need models that induce a representation, mapping the graph into an embedding and then predicting. The more ambitious step is to modify that embedding, like machine translation, starting from one molecule and generating an improved version. Property prediction already works reasonably well. They are now generating molecules and actually synthesizing them.

Barzilay started in natural language processing in 1997, at the moment statistical methods were arriving at ACL. Her English was still shaky. Half the papers were rule-based, heavy-linguistic approaches on small domains; the other half were the first corpus-based papers, simple by later standards: collect some statistics, then predict. She found it fascinating that one community could think so differently about the same problem. Her first paper had a single formula, no evaluation, just example outputs — and that was the standard of the field.

Statistical methods then dominated, and performance rose across basic tasks. The sad part, she says, is that linguistics largely receded; you have to hunt through a proceedings to find papers with serious linguistic references. Is that a loss? She pushes back. What does "explainable" even mean if the system performs the function? A calculator computes differently than you do, and that is fine. It is almost unfair to demand that a machine understand the way we do, because so many other information sources are absent when you train. Delivery is what counts.

In 1997, when she arrived in Israel, machine translation felt as impossible as flying to the moon, building a house in the garden, and living happily ever after. She could not imagine working systems within ten years. Now nobody is surprised to use translation daily — huge progress on a problem other methods never solved. Linguists tried writing syntactic trees and hunting for the right representation, and they did not get far; models using other signals did. She is not naive: change the domain or shrink the training set and models do "bizarre and funny" things. She treats that as a generalization problem, a technical question.

What bothers her is brittleness. Year after year in her NLP class, newspaper Hebrew translates beautifully; a Finnish recipe for Karelian pies that a friend sent is unreadable, because recipes are a tiny slice of the training data. Asked whether she would talk to a neural network for an hour, she says, "Oh, God, no." A visit to China showed her a chatbot that reportedly held four-hour conversations, which brought back Joseph Weizenbaum's ELIZA at MIT: string matching, almost no syntax, and a secretary who sat with it for hours. Weizenbaum stopped the work, horrified that a hint of understanding is enough for people to complete the rest. The lesson, Barzilay says, is less about how good the technology is than "how ready we are to believe that it delivers the goods."

Asked for a benchmark that is just out of reach, she does not name a flashier downstream task. She names few-shot learning. Papers that go from 55 to 65 with a handful of examples are not, in her view, doing anything realistically useful. The real threshold is learning from few examples in a way that generalizes, and becoming autonomous enough to find the dataset needed for a new task or a new language. That is where the field still is not. On "human-level intelligence," she distinguishes copying poorly understood human internals from delivering human-level functions across many areas, with the ability to keep learning. The latter is measurable, and that is what excites her: in her lifetime she has already seen huge progress across functionalities.

The other frame is imperfect human brains accelerated by stronger tools. She points to *Flowers for Algernon*, in which a miracle changes a patient's mind so that some things get much better and others much worse. Computer-augmented cognition, she says, is like cars taking us places we had never been: can we think differently, think faster? Fridman brings up Neuralink's brain-machine demo. She is more interested in a smaller fact: people often do not know their own attention has drifted. A gaze-based aid could tap you when you are writing a paper and thinking about the evening. She already knows how sticky measurement is. Her gym awards status points; she mocked them, then spent eighteen months protecting the status, running while injured, and cramming a "humongous" load into two days once she could run again, because she was not going to lose the rank.

Teaching is the last professional thread. Her large MIT machine-learning course, designed for CS majors, filled with non-majors as the field got hot, and many of them associated ML with struggle and failure. She launched a new class, "Machine Learning: From Algorithms to Modeling," aimed at majors and non-majors, and she questions whether a standard course really needs twenty classifiers. The barrier was not Python. It was math notation: big-O, linear algebra, probability, multivariate calculus — symbols that were simply not in visitors' lexicons. Interns came back amazed they could apply machine learning to problems she had never imagined; suddenly everything was a "miracle."

Advice for newcomers comes in two parts. Resources now exist at every mathematical altitude, so you can find a teacher who speaks your language; more important is to pick a domain you actually care about — curing cancer, self-driving cars, whatever — where there is data, you believe strong patterns exist, and we are not yet doing the work well. On the meaning of all this, she has no global answer. Each person has an inner set of things that matter; we are too busy conforming to hear it. When she was younger, missions were external: achieve this, be that. Now much of what she does is independent of recognition. "The most important thing for me is to be true to myself." Vanity still takes a minute. She finds her way back in solitude, and in books that pose the right questions.