The Hadoop Lesson That Made Google Give TensorFlow Away

Lex Fridman opens by calling the decision to open-source TensorFlow a definitive moment in the tech industry: a company of Google's size took a large project that lawyers might argue was full of IP and put it on the internet. The guest is Rajat Monga, an engineer and director at Google leading the TensorFlow team. By 2019 the library sat at the center of both cutting-edge research and large-scale production, and was already becoming an ecosystem of tools for cloud, phones, browsers, and specialized hardware, with TensorFlow 2.0 then in alpha.

Monga had been involved with Google Brain since it started in 2011 with Jeff Dean. DistBelief was the proprietary library; TensorFlow was the 2014 rewrite that went public in November 2015. Fridman wants the early mission, the open-source logic, and a question a lot of people still ask: what was Google after?

When Monga started, deep learning was intriguing and had shown early promise, but it had not yet taken off. The idea Andrew Ng and Jeff Dean were chasing was simple: take the research, scale it to Google's compute, put Google's data behind it, and ask whether bigger still meant better. That was the first year or two. "Can we prove that?" They got early wins, which he says is always great.

Two of those wins still get named. One was speech, in close collaboration with Google's speech research team, which was also getting interested. The other was images: the widely covered paper on a large unsupervised network that became famous for detecting cats. The mission was neural networks from the beginning. In 2011 and 2012 they scaled from hundreds of machines to thousands, and some runs even went to ten thousand. Google had been doing machine learning for a long time; deep learning was new. Scaling it showed that it could land in real products. Speech was first, then images and Photos, then many other products.

By 2014 the outside world had changed. Academia was pushing to do more with deep learning. Internally, Brain could see this was going to grow, and not just inside Google. "Maybe Google's ahead of where everybody is, but there's a lot to do." One company was not going to finish the work.

Fridman had just been talking with Chris Lattner about the same moment. For him, a large company giving away a core project full of IP, and in doing so telling the industry that open innovation was allowed, was one of the seminal events in software engineering. Monga says the initial idea came from Jeff Dean, and it sat on two arguments. The research argument was straightforward. Brain was a research group. They published. Sharing was how deep learning was moving so fast, and software was now the vehicle for that research. The academic libraries then in circulation, including Theano, were not in the same league as what Google had built.

The engineering argument is the one people outside Google often missed. Google had published papers on excellent internal systems, then watched someone else implement the paper and become the industry standard. Hadoop, grown out of Google's GFS and MapReduce work, was the exhibit. Google knew its own stack was better for a list of reasons, had invested heavily, and still found itself offering compatibility APIs on top of what everyone actually used, including Bigtable-shaped interfaces that were "not as good, but that's what everybody is used to." The question became: can we ship something better, actually help the community, and push a good standard forward?

They started the TensorFlow rewrite in summer 2014. By late 2014 they judged that open-sourcing was likely, so they designed for it. By then they already had Google use cases that shaped the architecture: large-scale data-center training, changing hardware including GPUs and the first TPU, and models already running on phones via handcrafted code they wanted TensorFlow to replace. DistBelief had been built for Google's internal systems in parallel with Theano, Torch (Lua), and Caffe; Yangqing Jia was at Google as well. When they got to TensorFlow they looked at those libraries and at DistBelief's limits. Research was moving fast, hardware was changing fast, and they needed to express "all kinds of crazy things."

The graph was not an accident. DistBelief already had a graph-like structure, closer to a straight line, more Caffe than a general graph. Production had always mattered: even with DistBelief they were deploying a lot. They did experiment with a more Python-native design that looked simpler to use. Without a graph, though, the next questions arrived immediately: how do you deploy, how do you distribute, how do you place work across devices? "That was probably what tilted the balance for us. And eventually we ended up with the graph."

Fridman asks whether anyone pictured 41 million downloads. Monga did not imagine that number. From a research perspective they saw a need, and a future where many more people would be doing this, but not that scale. The growth became visible after the open-source drop, when deep learning itself accelerated and TensorFlow sat in the right place to deliver what people wanted. Version 0.1, which they later mapped onto something like 0.5 or 0.6 in his memory, was a different project from 1.0.

What people loved first was the documentation. It was a huge step up from academic projects that did not think about docs. That changed the audience. Deep learning stopped being only a research activity. People who had "no clue what machine learning was" could pick it up and do something interesting. Over the next months the team planned for stability, then 1.0, because enterprises would not buy a moving target. Between the initial release and 1.0 came researchers, then hobbyists, then enterprises. Below 1.0, he agrees, you should not expect enterprise trust.

It is easy, in the middle of a fast field, to forget what enterprises want. Monga says people are still running models that are three or four years old. Inception is years old and still used by tons of people, and they are fine. They do not need the last bits of performance or quality. They want stability and things that just work. That is the value of making the older past simpler and more accessible. The research crowd wants the opposite: RNNs, then Transformers, then combinations with GANs and whatever arrived that week. The boundary keeps moving, but more and more of the past is usable.

Fridman guesses that the biggest real-world use case is taking something like ResNet-50 and doing transfer learning on a narrow problem. Monga splits the audience. For hobbyists, and for the apps you see on phones, that is the most common case, partly because it looks great on slides. Enterprises care about that too, but it is not the big thing. They have data they want to make predictions on. They used to do linear regression, logistic regression, and gradient-boosted trees. Some of that still benefits from deep learning, but structured data is the bread and butter. They also want the whole pipeline, TensorFlow Extended included, to be stable and simple, because they do not train a model once. They do it every day.

Fridman teaches deep learning and keeps meeting lawyers, insurers, and other older industries whose real blocker is that the data is not digitized. Monga gets the full range of questions, from "why would this beat my linear model" to "I have no clue where to start" to expert edge cases. His usual advice is not to chase the newest model. "Make something very basic work and then you can improve it." Organizing datasets, including TensorFlow Datasets, became part of the job because people asked for it.

François Chollet started Keras before he was at Google, first on Theano, then added a TensorFlow backend on his own when TensorFlow became popular, likely still before he joined. Monga did not find out Chollet was at Google until months after he arrived. Chollet was in research, writing papers, and maintaining Keras at nights and weekends. At some point the TensorFlow team noticed that people liked the API and that its author was in the building. They asked him to come over for a quarter to make the integration happen. His manager said a quarter was fine. Two years later he was fully on it.

The decision was bolder than it looks in hindsight. Google had been building its own high-level layer API in parallel, plus about three other community wrappers, because there was no standard. Users kept asking which one to use: a model lived in this API, another model in that one. For 2.0 they decided they had to pick one. Keras was what lots of people already loved, so they settled on it organically. Fridman notes that Keras had felt like a competitor. Monga's line is that it is interesting to put two things together that can align. Chollet, the team, and Monga all wanted the same thing: make it easier for a huge set of developers. In 2.0, Keras became the recommended path for beginners and for the basic enterprise cases, including transfer learning.

Fridman asks whether a project this large needs a benevolent dictator for life, the way Python had Guido van Rossum. Monga has always been involved in key design directions, but a lot is distributed. Martin Wicke, among others, has driven much of the open-source work and APIs. Over the prior year they had spent time on transparency, process, and special interest groups. At this scale, he says, they could not scale if every decision started with him.

The ecosystem is the point. TensorFlow.js made training in the browser a serious path; TensorFlow Lite covered mobile; SavedModel and TensorFlow Hub were how you were supposed to move a model from desktop to phone. Monga's short version of the mission: enable machine learning in two directions. Keep pushing the research frontier, including algorithms beyond deep learning. Then help everyone else take that research and put it in products. There are a crazy number of computers in the world, not just workstations and data centers. Phones and tiny chips already run models. The ecosystem is how you get ML onto every device with compute. TensorBoard was the first tool; TFX is the production pipeline. TensorFlow Agents and TensorFlow Probability started as research libraries and got used in production. Some pieces come from other Google teams, some from the community. They cannot and will not build every piece. They can enable other people to.

2.0's other job is making those pieces work together. When things look easy, that is the point: it should be easy for the end user, and a lot had to happen behind it. Hardware vendors still need a cleaner way in. TensorFlow started as a very monolithic system, and to some extent it still is. The core is large. Breaking it apart with clearer interfaces, on a four-year-old system that is still evolving at full speed, is "like changing the engine with a car running."

The downside of 41 million downloads is technical debt. It would be easier to start from scratch every few months. 2.0 does break some backward compatibility, but Monga says the conversion is meant to be straightforward, and the pressure not to throw the past away is real. If the user is a researcher who will not look at the code a year later, breakage does not matter. Production systems at Google and across the world run for a long time. Compatibility has a huge cost. The tradeoff, as he frames it, is that you might slow some kinds of change, but the overall value is bigger. You are not only refusing to break yesterday's user. You are telling tomorrow's user that this is how you do things, so they can come on board.

His method for designing new things is the reverse of letting the past drive. "When you want to do new things, you want to start with a clean slate design with a clean slate in mind, and then we'll figure out how to make sure all the other things work." You make compromises later. If you design while worrying about the baggage, "you'll never get to a good place." They were investing in conversion tooling, talked through at that week's developer summit, because a Python-2-versus-3 split was the risk everyone named. He expected people to move once they saw value in the new thing, and because the field itself was moving, new work would happen on 2.x.

Fridman, whose group had just switched to TensorFlow, asks whether Monga enjoys PyTorch. Monga's account is fair. TensorFlow chose graphs because production mattered, not just research. PyTorch said it only cared about research, skipped the graph, and made that part easy even if it was not as fast as it could be. They had the benefit of seeing what came before, including work on autograd. Competition made TensorFlow revisit a question they had already asked a couple of times: should we add dynamic execution? Eventually they said it looked like it could be done well, and that is how eager execution landed. Asked the Muhammad Ali versus Frazier version—would eager have arrived as fast without PyTorch?—Monga says they had tried variants before, so it would have happened, "but it might have taken longer."

What 2.0 really unlocks, in his view, is work behind the scenes. Graphs could always be fast if you spent time tuning. Cleaner APIs mean most people write code the same way, so the team can optimize that path out of the box, on one machine and distributed. The modular split matters for the organizations that actually change TensorFlow, not for a Pepsi that merely uses it: hardware vendors with custom chips, IBM in special interest groups, autonomous-vehicle companies with their own clusters and networks. Those groups can kind of plug in today. It is just hard. Cleaner interfaces let them evolve independently.

Fridman reads the numbers: 41 million downloads, almost 50,000 commits, almost 10,000 pull requests, 1,800 contributors. What does it take to build that? Monga wishes he could replicate it on demand. Timing mattered; TensorFlow grew with deep learning itself. After that it was listening, accepting external contributions, putting process in place, and a year of pushing transparency so people could see where the project was going. On a two-person project you do not think about that. At this scale, documentation, contributor experience, and tools are the work. The flywheel he likes is someone implementing a new architecture, putting it on GitHub, and someone else building on it.

On teams, he looks for cohesion. At this scale an individual can only do so much; the product of the team should be larger than the individuals added up. People have to care about what they are building. Google is bottom-up, research even more so, but a product this large also needs a shared direction so exploration does not spray everywhere. Superstars exist, at Google and on the team, and they can sit in tension with group dynamics. Exciting work makes that easier because there is room for many people to grow. Still: "Whether a superstar or not, they need to work well with the team." They might add a lot of value, "but if they're holding the team, then that's a problem."

Hiring uses Google's long-refined process. On top of technical skill, Monga cares about motivation. If that does not align, it will not work long-term for the person or the team, and it matters even for the most junior engineer: "However smart they are, it's going to be hard for them to succeed." Interviewers rate culture fit, not only puzzles. TensorFlow needs people comfortable with a fast-moving project and, now that it is a full-fledged product, people who will not cut corners. The hardest part of the job, he says, is striking balances: speed versus polish, whom to involve, when to say no. Deadlines help, as the Dev Summit and 2.0 alpha showed, but they try not to invent artificial ones. They iterate in the open, ship experimental work labeled as such, and treat a miss this month as next month's release.

Before TensorFlow, Monga led a team in Google search ads. Fridman frames ads as both a way to connect people to what they want and a way to ruin a page. Monga's version is that search ads were always treated as an extension of search: make the world's information accessible, including products. There is a minimum quality bar. If an ad does not hit it, it is not shown, even if that means losing money. He liked that. Advertising as a model is old and adapted to the web; pop-ups that stop you reading are the failure mode. The balance he wants is an ad valuable enough to the user that it can fund the service.

He does not think the internet is about to go fully paid. Ads will remain. But the last few years had already shown people willing to pay when they see value: Netflix, YouTube, news paywalls. He feels the shift in himself. The mix he hopes for is try something free, maybe with ads, then a clearer paid model for going further. That is also how he thinks about Colab. TensorFlow is open; you can run it on a desktop that keeps getting more powerful, or on a phone that is many times stronger than his first desktop. Cloud, for students, is about getting started: a website, no install, it just works. Free Colab is for exploring. Free can only go so far. Then you pay for more, the same logic as free versus subscription.

If you are a complete beginner, his path is the unglamorous one. Go to the website. Click tutorials and guides. Open them in Colab. No installation. Get started there. The whole conversation sits on the same pattern. TensorFlow's major choices were rarely the most elegant technical option. They chose a graph because they would not drop production. They chose Keras because the ecosystem needed one front door. They kept compatibility because tens of millions of downloads are real systems. They started splitting the monolith because other organizations needed to plug in. In each case, letting other people run things ranked above the beauty of the stack.