Chris Lattner on Mojo and the Search for Portable GPU Performance

Ron Minsky asks Chris Lattner how he came to compiler engineering and language design. Lattner recalls growing up with early PCs, typing programs from books and learning to debug the mistakes he made. Games and early online discussions sparked his interest in graphics, performance and hardware.

At the University of Portland, professor Steven Vegdahl's enthusiasm for compilers drew Lattner into the field. At the University of Illinois, he built LLVM. He liked the way a compiler course involved constructing successive layers of a larger system rather than completing isolated assignments.

Lattner says the compiler class also made abstract data structures useful in a concrete system. Minsky compares that experience with an operating-systems course, where students likewise build one layer on another.

Minsky notes that Lattner has moved between compiler engineering and language design. Lattner says his career followed changing interests: LLVM taught him about low-level code generation, then helped enable work on a C++ implementation and later the higher-level Swift language. He sees his most useful work at the hardware-software boundary.

Lattner describes moving upward from backend code generation to Clang and Swift. Working with AI systems and an iPad app for teaching children to code also reflects his desire to pursue useful problems across different levels of the stack.

Lattner says LLVM began in 2000, and he joined Apple in 2005. By around 2010, Clang's C++ support could build itself and advanced libraries. Implementing C++ was technically interesting, but after shipping it he was burned out and wanted a better language.

Swift began as a nights-and-weekends project, not a management directive. Lattner argues that language designers need not invent every idea: pattern matching, for instance, was already well established but absent from many mainstream languages at the time.

Minsky explains the usefulness of algebraic data types and pattern matching: they let a program represent alternatives and analyze cases reliably. Lattner says he approaches these features as an engineer, drawn first to the problems they solve and the expressive code they enable.

Lattner says pattern matching made an immediate impression when he learned OCaml: it made ordinary tasks simple and expressive. He believes adoption depends on demonstrating utility, even when formal type theory explains why features compose correctly.

Minsky adds that strong mathematical foundations help features generalize and compose beyond their original uses. Lattner agrees that good designs fit together simply, even if his own process sometimes rediscovers ideas already known to researchers.

Minsky asks how Lattner seeks mathematical insight after approaching a design as an engineer. Lattner says he follows research and reads the explanations that connect ideas to examples and results, even if dense mathematical notation is less comfortable for him.

Lattner says he often learns from papers' examples and results or from accessible blog posts about academic languages. He values explanations that connect a sophisticated feature to a concrete problem. Minsky agrees that informal explanations complement rigorous theory.

Lattner compares language complexity to a grain of sand that forces many complications elsewhere. C++ has enormous existing constraints and cannot simply remove old features; Swift and LLVM also contain design decisions he would revisit. Starting a new system offers a chance to learn from those choices.

Minsky asks about Mojo and Modular. Lattner traces the problem to a shift from CPUs to GPUs, TPUs, FPGAs and other AI accelerators. He describes MLIR, developed from lessons learned with LLVM, as foundational compiler technology for this newer and more varied world.

LLVM helped provide a common foundation for CPU-era languages including Swift, Rust and Julia, Lattner says. MLIR applies lessons from that work to a wider range of accelerator and AI compilation problems, although a shared intermediate technology has not eliminated fragmentation.

Despite MLIR's wide use, Lattner says AI software remains fragmented across stacks such as NVIDIA CUDA, Google XLA and AMD ROCm. High-level tools such as PyTorch must operate on top of incompatible foundations, while hardware and models continue to change rapidly.

He attributes part of the fragmentation to incentives. Hardware vendors must build software for their own chips and upcoming roadmaps; they rarely have the resources or incentive to build a common layer for competing hardware. Users, by contrast, want portable software across devices.

Minsky notes that high-level frameworks can also struggle with incomplete cross-platform support because they sit on incompatible lower-level systems. New hardware and software arrive quickly, adding to the difficulty of keeping the ecosystem coordinated.

Adding another abstraction above inconsistent stacks cannot necessarily repair their performance or reliability problems, Lattner argues. When something breaks, users still have to debug the layers underneath. Modular set out to build a lower-level, portable successor to the hardware-specific software foundations.

The goal, he says, was deliberately difficult: match or exceed vendor software on the vendor's own hardware, despite NVIDIA's experience and long CUDA head start. His argument is that someone must do that work if users are to have an ecosystem less dependent on a single vendor.

Lattner describes this as a long-term investment rather than a quick fix. He hopes that a portable foundation will help more people use accelerators and reduce dependence on one hardware supplier.

Minsky asks how Modular can build a shared platform when selling programming languages is notoriously difficult. Lattner says the company does not sell Mojo itself: the language is free, while the intended business is enterprise services built on the foundation.

He describes platform teams operating hundreds or thousands of GPUs while product teams continually introduce new models. Different hardware, CUDA versions and rapidly changing open-source dependencies make production difficult. Modular hopes to help those teams with performance, observability, manageability and deployment across hardware.

The intended enterprise value is not just a faster kernel. Lattner says coordinated support for devices and software releases can reduce the number of combinations a platform team must manage, making it easier to keep production workloads running.

Lattner describes three layers: Mojo as the programming language, MAX as a single-node high-performance serving layer, and Mammoth as a cluster-management layer. He says he initially resisted creating a new language because languages are costly to build and hard to get people to adopt.

The problem, in his view, was how to write accelerator code that is portable without giving up the full performance of expensive hardware. Existing languages did not provide a satisfactory answer for the heterogeneous devices and specialized numerical formats his team wanted to support.

The conversation turns to hardware. CPUs have many cores and support established threaded workloads. GPUs expose a different model, with streaming multiprocessors and many threads, while tensor cores add specialized matrix-multiplication hardware for the operations common in AI.

Minsky describes GPU warps as groups of threads executing together, with hardware resources that make switching among threads cheap enough to hide latency. Tensor cores introduce another, specialized execution model whose performance and memory characteristics change across GPU generations.

Minsky describes how NVIDIA hardware changed from Volta through Ampere and Hopper, including differences in tensor cores and the memory model. He and Lattner emphasize that a kernel must account for such changes rather than assuming one fixed GPU architecture.

Lattner argues that CUDA was designed for a simpler generation of GPUs and has struggled to evolve with this hardware. TPUs and other accelerators can depart further from the GPU thread model by organizing hardware around large matrix operations. His stated goal for Mojo is to support such systems from first principles.

He also points to incompatibilities across hardware generations and argues that a tool that trades away too much peak performance can leave users needing a second system. His aim is to combine usability and portability with enough control to approach full hardware performance; these are goals, not demonstrated universal guarantees.

Minsky contrasts low-level GPU languages in the C++ tradition with safer alternatives. Undefined behavior gives compilers latitude to optimize correct programs, but mistakes such as uninitialized variables can make real programs difficult to reason about and debug.

Lattner sympathizes with C++ implementers because a vast body of existing code must keep working. Optimizers often have to improve programs they cannot change. In new AI workloads, he argues, programmers often do control the code and would benefit from explicit power and expressiveness rather than heroic benchmark-specific compiler tricks.

Lattner says legacy constraints are understandable, but a new language can reject mistakes such as use of an uninitialized variable with a clear compiler error. He sees usability and diagnostics as part of Mojo’s design, alongside raw performance.

Minsky observes that GPU kernels are often small, high-value programs written by a relatively small set of experts, making precise performance control especially important. Lattner adds that AI coding tools can help newcomers learn a language or translate existing kernels, while cautioning that the tools are not magic.

Asked about Mojo's design, Lattner says it starts from Python syntax because much of the AI community already knows Python. It diverges when needed for performance, explicit control and metaprogramming. His goal is to enable powerful hardware libraries in the language rather than hard-code every abstraction into the compiler.

Metaprogramming lets code run at compile time as well as runtime. Many languages provide different mechanisms for the two stages, such as ordinary loops at runtime and recursive C++ templates at compile time. Mojo aims to let programmers use similar language constructs in both places.

The ability to use lists, strings and algorithms at compile time allows library authors to construct higher-level abstractions before the program runs. Lattner argues that sharing a language across stages can also improve debugging compared with complex template stack traces.

Lattner says this matters for tensor cores because layouts and indexing differ by vendor and device generation. A library can represent those details at a higher level and perform substantial work at compile time, while developers use familiar language and debugging tools.

Minsky frames metaprogramming as an old solution to interpretation overhead. A generic interpreter can express a domain-specific computation, but it makes decisions at runtime. Code generation can instead carry out known decisions ahead of time and produce specialized machine code.

The difficulty is making that machinery understandable. C macros and C++ templates can be powerful but painful to reason about, especially when errors appear only after deep expansion. Minsky describes Mojo's aspiration as one language and type system spanning generated and runtime code, with explicit staging.

Minsky says his team's early experience with Mojo has been promising. Compared with C++ templates and Python-embedded GPU DSLs, he appreciates seeing from the syntax whether a value belongs to compile time or execution time. That explicitness is useful for performance engineering.

Minsky notes that PyTorch can make Python code appear straightforward while carefully delaying operations and overlapping CPU and GPU work. That can deliver excellent performance, but its execution model is more subtle than the source may suggest.

They discuss the appeal and danger of a sufficiently smart compiler. Automatic parallelization or lazy execution can make simple examples look easy, but a small code change can defeat an optimization or make performance collapse. When that happens, users may have to inspect compiler internals.

Minsky compares CPUs, which use speculation and prefetching to make sequential programs run quickly, with more explicitly parallel accelerators. CPUs are flexible and their hardware tricks often work remarkably well, while explicit machines demand more from programmers but can offer performance and power-efficiency advantages.

Lattner says explicit control is a deliberate design choice for Mojo. In his account, it can make advanced hardware programming harder than a purely automatic interface, but helps users understand and pursue the performance they need.

Minsky asks how metaprogramming delivers portable performance. Lattner says even a single matrix multiplication algorithm has many specializations: data types change how much fits in cache, matrix shapes change access patterns, and hardware imposes different layouts and operations.

A change from 32-bit to 16-bit values can double the number of elements that fit in a cache. The best access pattern may therefore change too. Matrix shape adds further cases, so even an apparently simple generic operation can require substantial specialization.

Those differences occur within a single vendor's product line as well as across vendors. Lattner describes a layout tensor abstraction that represents not just a multidimensional array but also its memory layout and access pattern, allowing libraries to map the computation onto particular hardware.

His proposed combination is a type system, compile-time metaprogramming and domain experts who understand the hardware. Rather than embedding every optimization in the compiler, he wants kernel specialists to build reusable libraries that specialize for both the computational problem and the target device.

Minsky summarizes the benefit as a coherent program whose compile-time portions select and generate hardware-specific code. He then raises two concerns: whether abstractions can hide enough complexity for people to reason modularly, and how to test the potentially enormous space of configurations.

Lattner's response centers on types and traits, analogous to Swift protocols, Rust traits or Haskell type classes. A trait can describe operations such as iteration across different collection types, allowing generic code to be checked against an interface without instantiating every concrete implementation.

An iterator is a simple example of a shared interface: linked lists, arrays and other sources can implement the same concept. Generic code can use the interface while the compiler checks that the required operations are available.

Minsky says C++ templates' expansion model can make errors difficult to understand when deeply composed, and repeated generation can be expensive to compile. Type-based generic interfaces can improve reasoning and composition. The exchange does not provide a complete test strategy for every hardware configuration.

Lattner says Mojo's compiler first parses generic code without fixing the target's pointer size or architecture. That representation can then be specialized for a CPU or GPU. A GPU-targeted portion must ultimately be packaged as a self-contained piece of code for that device.

Minsky asks whether every executable must include compilers for all targets. Lattner says a normal build produces a conventional executable, while a Mojo package can retain portability for later specialization. He compares the broad idea of target-independent representation to Java, while stressing that the implementations differ.

Minsky draws a parallel with NVIDIA's PTX, a portable intermediate representation compiled for particular GPUs. Lattner distinguishes Mojo's representation by its richer generic type information. Minsky adds that preserving programmer-directed specialization gives developers more control over how target-specific code is produced.

The distinction also concerns control. In the PTX model described by Minsky, target-specific compilation is largely hidden from the programmer. With Mojo’s retained generic representation, the intent is to preserve instructions about specialization until the target is chosen.

Minsky asks how a language can remain as approachable as Python while adding the precise types needed for safety and performance. Lattner says there is no perfect answer, but Python's syntax gives the team a starting point and a reason to resist adding features without sufficient need.

He wants core mechanisms such as metaprogramming and checked generics to absorb complexity rather than accumulating unrelated features. He credits Go's restraint as an example: choosing a smaller feature set can keep complexity down, even though every user may want one more feature.

Lattner recalls criticizing Go for initially lacking generics, but says its designers maintained a boundary and later added the feature deliberately. The lesson he draws is that every individually useful feature still carries a complexity cost.

Minsky calls the challenge maintaining a language's power-to-weight ratio. Teams need room to try an idea, retract it and adjust, but that becomes harder once a large open-source ecosystem depends on the language. Lattner says Swift's open proposal process brought valuable ideas as well as pressure to add features.

For Mojo, he favors a small core team making deliberate decisions. He says the team is discussing what a 1.0 release should mean, without committing to a date, and is interested in versioning that permits language changes while allowing packages from different major versions to interoperate.

Lattner contrasts that aspiration with the long Python 2-to-3 transition, where the package ecosystem had to migrate. He points to C++'s ability to link code built under different language editions and hopes Mojo can combine compatibility with tools that help migrate source code.

Minsky observes that a strong type system can help migrations by identifying errors without executing every path. He says coding agents have also worked surprisingly well at following type errors after a change. Lattner agrees and points to useful ideas in Rust's approach to stabilizing APIs.

Minsky asks how AI agents work with a newer language such as Mojo, which may not have much representation in model training data. Lattner says publicly available Mojo code and GPU kernels can be indexed by coding tools, and that configuring tools to access the repository makes a substantial difference.

The conversation then turns to Mojo's relationship with Python. Lattner describes performance-sensitive CPU and GPU code as a current use, and says Python-extension support is available in preview. His goal is to let users move slow portions into Mojo without the usual complexity of writing bindings to another language.

At the time of the discussion, Lattner describes the Python-extension workflow as preview functionality. He says users can move a slow part of a Python package into a Mojo file while keeping a familiar syntax, rather than necessarily rewriting an entire application.

He reports users obtaining substantial CPU speedups in some cases and then moving work to GPUs. These are examples he offers, not a general benchmark. Minsky notes that many Python packages already depend on extensions written in C, C++ or other languages.

Lattner outlines possible future additions, including more generic capabilities, runtime traits and Python-like classes. He speculates about Mojo becoming useful for broader application programming and eventually feeling more like an evolution of Python, while acknowledging that it is not Python-compatible at the time of the conversation.

He emphasizes that the larger vision remains unproven: he has confidence from previous language work, but asks users to assess Mojo based on present needs such as performance code, GPU kernels and Python extensions. Minsky agrees that even the near-term goal of improving heterogeneous compute is ambitious.