Python Isn't Finished: Chris Lattner on Mojo and the Hardware Mess

Two years after their last conversation, Chris Lattner is back on the Lex Fridman podcast with a new language. Fridman introduces him as one of the central engineers of modern computing: LLVM and Clang, the Swift language, key work on TensorFlow and Google's TPUs, a stretch as vice president of Autopilot software at Tesla, and years at Apple. The new work is Modular, a stack for distributed training, inference, and deployment, and Mojo, billed as a superset of Python with Python's usability and, in many cases, C and C++ performance. Fridman notes that Mojo code has demonstrated more than 30,000x speedups over Python; later they will unpack a 35,000x figure in detail.

Lattner refuses to start with syntax. He has spent years on LLVM, mobile, and servers, and he argues the world has changed: new GPUs, machine-learning accelerators, and ASICs now make AI go fast. At Google he worked on TPUs, among the largest-scale AI systems deployed. The incoming compute platform is not just CPUs or GPUs or TPUs or NPUs or IPUs — "all the PUs." For software people, hardware you cannot actually use is not a gift. Having the theoretical ability to program a chip and unleashing the world's smart people on it, he says, can be quite different.

Mojo came from a more specific problem: take the infrastructure under machine learning and make it accessible to researchers who are not themselves GPU experts. Along that path the team realized they needed syntax. They needed a programming language. Modular, founded a little over a year before this taping, is trying to up-level that stack so more research reaches production. TensorFlow and PyTorch grew into systems that power much of the industry, but they were not designed with current demands in mind. Where were large language models eight years ago? They did not exist. On the other side, every large company seems to announce a new chip. "Moving system on one side, moving system on the other side, and it just turns into this gigantic mess."

Fridman, fully in jest, calls one of Mojo's main features the right to use a fire emoji as a file extension — the first emoji extension he has ever seen. Why, in the 21st century, are file extensions not Unicode? Lattner treats the joke as a product decision. File extensions are metadata, so why spend screen space on them stacked next to gray text and PDF files? If you are going to do something cool, you want it to stand out. Emojis are colorful, visual, and beautiful. You can still use `.mojo` if you prefer.

Support is mostly already there. GitHub, Visual Studio Code, and Windows File Explorer handle the flame because internationalization put Unicode in ordinary paths. The one snag Lattner has seen is command-line Git, which treats the emoji as unprintable and dumps hex. He has faith Git can be improved. The public reaction, he says, sits on a spectrum from "that makes sense, cool" to "you're killing my baby" and "how am I going to type this?" They do not have to bet the whole farm on the emoji; both paths can exist.

Emoji in identifiers is not even new. Swift allowed it; an Apple demo used the dogcow, and the internet immediately named functions after a pile of poop. Lattner is fine with that as long as it survives code review. The detail is small, but it is the tell: he is willing to spend taste on a millisecond of daily experience, which is how you design a language as a product rather than only as a specification.

Is Mojo only for AI? Lattner says it is AI-first: Modular needs it, and AI is driving the requirements. The solution to those problems is not actually AI-specific, so they are building a general language. Many Python libraries today are a Python API over C and C++ underneath. That pattern is true in AI and in scientific computing, data processing, and a lot of other domains. Anywhere you see it is an opportunity for Mojo to collapse two worlds into one thing.

Fridman asks what he loves about Python. Intuition, he says: it feels like writing English, other people's code is easier to read than C++, and the ecosystem has a package for almost anything. Lattner adds that machine learning helped Python grow when TensorFlow and PyTorch embraced it. The deeper property is that Python is "the universal connector." It lets you compose systems without understanding how each one works. Usability and ecosystem feed each other until the loop is hard to displace.

Then the problem. You could complain about several things, Lattner says, but people usually complain that it is slow. Other people would complain about tabs versus curly braces, "but I mean, those people are just wrong." The rest of the conversation is an attempt to keep the connector and delete the tax — without asking the world to migrate off Python the way it had to migrate from Python 2 to 3.

Fridman has just left Emacs for VS Code and caught hate mail for even considering Vim. Lattner takes the opening. Indentation, he argues, is better than braces, and he can defend it without calling it beautiful. Python has millions of programmers and owns machine learning. Brace languages already run formatters, so you end up with indentation *and* braces. If grouping is the point, keep one thing and delete the clutter. Python-style indentation also sidesteps the swamp of omitting braces for a single statement.

Fridman wants statistics on bugs from bad indentation versus C++ formatting mistakes. Both suspect the number is small once an editor like VS Code indents for you. You do not fight the tool; the editor shows you how the code will run. Lattner knows it is a religious fight and likes poking the bear. Then he comes back to the thesis that actually matters: Mojo is meant as part of the Python ecosystem. They are not looking to break Python, change it, or "fix" it.

"We love Python for what it is. Our view is that Python is just not done yet." One reason it is slow is that it lacks the features you need for C-like programming, so people get forced into C. Mojo's bet is to add those missing features so you can keep everything people love and not get kicked out when the work gets computationally intense, hardware-y, or weird. Later he will refuse even to joke that Mojo is Python 4.0. The 2-to-3 migration left too much scar tissue. The posture is finish Python, do not replace it.

Is Mojo compiled or interpreted? Lattner gives a complicated answer: it is interpreted, JIT compiled, and statically compiled. Python's beauty is dynamism, including metaprogramming. Define `__add__` and plus works on your class. PyTorch automatic differentiation and eager mode live on that runtime magic. Those features will not run on an embedded processor or a GPU, or you would not want them to. So Mojo takes the interpreter Python already has and lets it run at compile time.

The payoff is compile-time metaprogramming with Python-style APIs. Overloaded operators and the dynamic patterns inside frameworks can be lifted so they run while compiling, then emit code for the accelerator. C++ templates, in Lattner's reading, became powerful by accident; they were not designed to be Turing-complete. They also live in a different universe from runtime code: different syntax, different concepts. Zig is exploring a similar idea — run at compile time what you can run at runtime. Lisp already had the right shape. Metaprogramming and programming are the same, so a library works at both times.

Pulling it off is "super hard." Under Mojo is a new compiler design on MLIR, with caching, interpreters, and JIT compilers — an interpreter inside the compiler. That is what later makes autotuning possible: the compile-time interpreter can take your code, run candidates on the target machine, pick the winner, and stitch it back into compilation. The compilation process itself becomes a machine-specific search, which is rare in classical compilers. Fridman, a Lisp person, lights up. For machine learning, Lattner agrees, metaprogramming is extremely useful.

A PyTorch or TensorFlow model is an abstract specification of a compute problem that then gets mapped in many ways: run it at batch size a thousand, spread it across a thousand GPUs. That mapping is why machine learning became a metaprogramming problem. Autotuning is Mojo's concrete answer. Lattner says almost none of this is original research; they are remixing known ideas. Hardware people know cache sizes, register counts, vector widths, GPU SM counts, warp sizes, and that a TPU's tile size is 128. His belief is that most programmers do not actually want to know this stuff.

Those magic numbers also move. An A100, an H100, and a Volta want different parameters for the same algorithm. Code written on a laptop may later run on a server with a different GPU. The old answer was `#ifdef`s and an exploding test matrix. Autotuning says there is plenty of compute: describe the algorithm — carve it into blocks, parallelize, cut this way or that — then let the compiler try candidates on the real machine, cache the winner, and reuse it. Set it and forget it; when the machine changes, it can try again.

Ultimate performance matters for more than bragging rights. For an enterprise it is fewer servers; for the environment it is efficiency. Lattner jokes that you could say Python is bad for the environment, and moving to Mojo is at least 10x better out of the box. Performance also makes better products. Drop model latency and a product team can make the model bigger, which the customer actually feels. In this industry, capability is often gated by inference cost, not by the next algorithmic idea.

Fridman asks how Mojo can claim a 35,000x speedup over Python. Lattner says he can even do more, but start with CPython — the interpreter you get when you type `python3`. Interpreters read, parse, and interpret bytecode, and that extra layer is slow. Moving to a compiler and getting the interpreter out of the loop is a 2x to 5x to 10x speedup depending on the code. That is just modern technique, before any heroics.

CPython also boxes everything. An integer lives in an object with a header and payload, so passing it is passing a pointer. Allocation, pointer chasing, and reference counting all tax modern CPUs. Getting values out of the heap and into registers is another roughly 10x. People complain about the GIL; Lattner notes it is tightly intertwined with reference counting. You cannot throw one away and keep the other. Then come vectors — 4, 8, 16, or 32 operations at a time — plus threads and control of the memory hierarchy, none of which Python exposes.

Stack those layers and the outrageous number becomes a product of named techniques. Mojo's library gives relatively small syntax for big weapons against slowness; you can ask to `parallelize` a loop. Lattner says people have been having fun just taking code and watching it go fast. The adrenaline is real. The engineering claim is that each multiplier has a cause, and together they are how you stop treating Python's slowness as a fact of nature.

Python has types — strings, integers, dictionaries — but they live at runtime, so you do not have to spell them. That forces a consistent representation: a pointer to an object with a header, and the same dispatch path for every method. Mojo keeps fully dynamic Python objects, then lets you say a value is a 32-bit integer or a 64-bit float so the compiler can optimize and complete code. You add types progressively, wherever you want. If you do not want to deal with it, you do not have to. "It's not that types are the right thing or the wrong thing, it's that they're a useful thing."

Fridman sighs that adults use strict typing and he has been in mild rebellion. Lattner pushes on the cultural pressure: Python as a guilty pleasure you must rewrite when it gets serious. People rewrite to deploy, for performance, predictability, or a tiny server with no dependencies. What if Python could hit those goals? A huge internet company may have a house rule that you use types so humans can talk to each other. That does not mean everyone must use them always. Because Mojo is a compatible superset, it has to work without types, with all the dynamic behavior, packages, and list comprehensions. Lattner does not want the world to repeat the Python 2 to 3 migration.

Python's annotations grew from a fragmented world of external checkers. In the language spec, types are ignored — you can put any expression in a type position — which is extensible and also muddy. CPython cannot use them for performance; they are hints. In Mojo, if you declare a type, it is that type, checked at compile time. Numeric types are not magic in the language: structs are static, classes stay dynamic, and integers, floats, buffers, and tensors can live in the library so they are not second-class. In May 2023 he calls Mojo a 0.1, released early, with a public roadmap. The Discord already had more than 10,000 people two weeks after launch. There are proposals for braces. They tell people no.

Another tool is immutability: `var` versus `let`. In Python you pass a pointer to an array; a callee can stash it, and later someone else mutates your data. Mojo does not force a solution, but it enables value semantics. Collections, tensors, and strings behave like proper values, as if passing them copied the data. The trick is providing copy behavior without copies. You increment a reference on pass; you copy lazily only if someone mutates while another owner still exists. Implemented right, that defines away bugs and often reduces copies. PyTorch's `clone` on a tensor is the older, easy-to-forget version of the same idea.

The language's job is to give type authors low-level hooks — ownership, drawing on work in Rust, Swift, and C++ move constructors — without forcing the model on every Python-shaped program. Default lists stay reference-semantic for compatibility; a value-semantic list is a separate design. Argument conventions matter for the weird types systems programmers live with: atomics, mutexes, uniquely owned database handles that cannot be copied or even moved. Mojo has borrowed (read-only) and mutable-reference conventions so those types can exist without becoming the default for typical Python users.

Lattner sees a spectrum from poking bits in memory up to application scripting, and the goal is one system that scales. Fridman's distrust of concurrent mutation leads him to Moore's law giving way to multi-core, then accelerators, then specialization. Machine learning people invented tensors as an algebraic abstraction over a huge parallel dataset. Because TensorFlow and PyTorch express the math, not the byte placement, the industry could map work onto parallel machines and explode AI compute. That same abstraction is what you need if you want deployment across many machines without drowning in copies and races.

Modular's compute platform, the company copy says, dynamically partitions models with billions of parameters across machines. Lattner walks the recent history. TensorFlow was about graphs and relatively good at deployment; PyTorch won on usability and is weaker at deploy time. Researchers who are brilliant at architecture and data do not want to know C++ or hardware. They train, throw the model over the fence, and a deployment team throws it back because it is too slow, does not fit, or crashes. Each toss takes weeks of project managers. Getting a model to production in weeks or months is not atypical. He has heard vice presidents ask why they need a team of 45 people.

Python often does not deploy. Teams rewrite into C++, then carve a model across ten machines because it will not fit on one. TensorFlow and PyTorch were not designed for LLMs; they come from a ResNet-50 and AlexNet-on-one-GPU world. Modular's "bitter enemy" is complexity: hardware software stacks, a research landscape that is not done, cloud networking, and a serving world of zero-copy and async I/O that barely reached machine learning. The joking mission is "to save the world from terrible AI software." They would not have built a language if Python were already good enough.

The old world was CPU and CUDA, a dense layer kicking off a CUDA kernel or an Intel MKL path. Then hardware multiplied and models grew to about 2,000 operators, with perhaps thousands of chips if you count phones. Kernel fusion is a real win — do a matmul and a ReLU in one pass so you do not round-trip memory — but fusing every activation creates a permutation explosion. Compilers help generate kernels; they also exclude people who know analog computers or sparse numerics and should not have to patch a compiler. Mojo tries to put programmability back so those people can extend the stack.

Heterogeneous means many kinds of computers at once. A CPU can preprocess while a GPU multiplies; a modern phone has big.LITTLE cores, GPUs, neural accelerators, and media blocks that all need choreography. That is a miniature data center. Placement is scheduling: cost of moving data versus cost of running on a given block. You can start with manual knobs, then heuristics, then autotuning or even reinforcement learning over a space too big to search exhaustively. You cache the answer so you do not pay that search every time a model runs. Lattner later says Mojo did not start from making Python 2x faster. It started from what the hardware can do — the speed of light — and worked backward. The bottleneck is usually memory.

Mojo is not done, Lattner warns, but people already move small Python fragments over unchanged and see about 12x. The north star is a full superset. Someday packages should run without CPython; today Modular is unwilling to wait. They built an integration layer that imports arbitrary existing packages — including the ones on your disk — through CPython, so you get compatibility now and Mojo speed only on the code you migrate. The two-world problem is painful: a Python debugger that cannot step into C, wrappers around the interpreter's reference-counting rules, hybrid C extensions. Collapsing that into one world is the point.

The analogy is C and C++: move code over, then adopt features as you wish. They will not cheat on the long tail. Arbitrary-size integers are annoying if you want registers, and skipping them would be easier, but then you are not a superset. Lattner tells a Clang war story from around 2006. He wanted a beautiful standard-compliant parser, then `#include <stdio.h>` pulled in all the GCC extensions. Implementing yucky compatibility is easier than asking every C programmer to rewrite code written by someone who quit ten years ago. Microsoft's CPython 3.11 work — about 15 to 20% across the board — is "awesome" and composes with Mojo. It also does not run on GPUs. "20 percent's good. 35,000 times is better."

He talked with Guido van Rossum before launch. Guido's concern was fragmentation after Python 2 to 3. Lattner leaned on Swift's playbook: do not rewrite a million-line Objective-C app; write the next class in Swift, integrate deeply with the existing runtime, and later vend Python-facing interfaces so packages like NumPy can move without abandoning clients. He will not call Mojo Python 4.0 — "that might be a little bit far even for me." He sees Mojo as a member of the Python family, not a raid on Guido's legacy. Modular's engine, similarly, goes under TensorFlow and PyTorch rather than replacing them. People do not want to rewrite years of models.

Why not Swift for machine learning? Lattner left Apple in January 2017 to do AI, landed at Google around TPUs, and ran Swift for TensorFlow as research. Some ideas, including graph-program extraction from Python bytecode that he discussed at an LLVM conference in 2018, showed up in spirit in PyTorch 2. The project itself stumbled because Swift is not Python and TensorFlow at the time was not set up for eager mode. Julia shares ideas and an LLVM heritage; he underestimated that part of Julia's mission was end-to-end Python-like workflows, and the Julia community's response at Mojo's launch surprised him. Julia still would not be Python even with more money. "Meet the world where it is." Python sticks because of packages, low startup time, notebooks, glue-like objects, and a teaching loop that now includes his kids.

Two weeks after launch, Mojo Playground runs in cloud notebooks because the team wanted crash monitoring before a local download. Signups were running about one person a minute, past 70,000. Swift's 2014 WWDC launch is the scar: years of secrecy, then overnight production use, bugs, broken source compatibility, and technical debt. This time they labeled Mojo 0.1, told people not to use it in production, and refused to race. Jeremy Howard — who once beat a Google TPU team on DAWNBench with progressive image sizing, and whose site says "making AI uncool again" — has wanted this stack for years. Lattner jokes that Mojo exists because he finally listened. People have already tweeted Game of Life going 400 times faster.

Language details are still landing. Function overloading exists because typed languages need different behavior for integer versus range subscripts, while Python objects are dictionaries keyed by a single name. `def` stays Python-dynamic; `fn` is a strict mode that requires declared variables so typos fail closed. Errors are implemented like Swift, Rust, and Go — returning a variant — not C++ "zero-cost" exceptions that make throws thousands of times more expensive and blow up binaries. That design can run on GPUs. Mojo also destroys values after last use instead of at end of scope, which helps memory, tail calls, and ownership. Traits, classes, and richer packaging are still on the roadmap. Mojo itself is about seven months old.

Building Modular is Lattner's first company from scratch, which he finds liberating: the compiler is the product, not accounting software for someone else's revenue. Co-founder Tim pushed culture early. They hire specialists who already have big-tech jobs by paying well, going remote-first, and flying people together because Zoom does not replace the lunch table. The Apple lesson he repeats is work backwards from customer problems, not from cool technology. iPhone 1 shipped without copy and paste. Balanced teams beat a room of identical 10x clones. He wants an inclusive culture with a clear vision, not amazing people fighting each other or waiting to be told what to do.

Large language models, he thinks, will crush rote work and LeetCode-shaped tasks and will not end coding. Hallucinations help brainstorming and hurt production correctness; he would rather see algebraic, proof-like systems under the model. He does not worry about Skynet — "then I'll be dead" — and is not betting on AGI next year. Humans adapt slowly; robotaxis were supposed to be solved by 2020. The future of programming he wants is less complexity and more people who may not call themselves programmers. Excel is already the most popular language. Advice to students: do not memorize the book cover to cover. Go build something you wanted to exist. If everybody goes left, it can be powerful to go right.