Python Is a Recipe Written for Two Readers

Python 3.11 was about to ship with a claim that would look flashy in any engineering field: CPython, the team said, would run 10 to 60 percent faster than the previous release. Lex Fridman asked Guido van Rossum how they pulled that off, and what CPython even is. Guido did not start with the speedup. He corrected the noun. “CPython is the last Python implementation standing,” he said, “also the first one that was ever created” — the original interpreter he began more than thirty years ago. The *C* in the name is the language it is written in. A programming language’s default implementation is a program in another language, which sounds strange to outsiders and obvious to programmers. This episode, Lex warned, would have to speak to both.

Lex split the audience into a thought experiment: two people on a boat in the middle of the ocean. One has never heard of programming and is into fishing. The other is a Silicon Valley engineer who has written C, C++, Python, Rust, and Java and can recite language history. Guido chose the fisherman first. “He seems like the most useful in the middle of the ocean,” he said — you have to keep him happy. That listener owns a phone, is suspicious of what goes on inside it, and has at least heard that a tiny computer lives in there. A programming language is the code that tells the computer what to do. Zeros and ones, assembly, registers: Guido would not go there. When people talk about human language, they do not start with vocal tracts and tongue position.

His working definition: a programming language is a list of instructions, like a cookbook recipe. Acquire a loaf of bread, slice it, take two slices, put mustard on one and jelly on the other, then the meat, then the cheese. Science teachers, Lex noted, like to play a cruel version of that game: students write sandwich steps, the teacher follows them literally until the class is forced into unambiguous English. That game is the gap between natural language and code. Ambiguity is a feature, not a bug, in spoken language — the dance of communication. Lawyers treat it as a feature they can bill. Everyone else works around it with context. Say “I love you” to someone who matters and they do not compile the sentence and return `please define love`. “I imagine that my wife and my son interpret it very differently,” Guido said, even though it is the same three words.

Lex turned to PEP 8, the style guide, and the line generations of Python programmers have memorized: readability counts. Code runs on machines that do not care about spaces or a swear word in a comment. So who is readability for? Guido flipped the picture. What emerged from the late 1940s onward is that software is a social activity. The mad scientist alone in a lab writing brilliant code is a myth. Even that recluse cannot type fast enough to finish a program while still remembering what the first lines meant. A cookbook author writes once, prints a hundred thousand copies, and aims at the amateur in the kitchen. A program has two audiences at once: the computer, which needs precision, and other programmers — including you, a year later — who will have to debug and change it. Cherry pie recipes are not that complex. Software is, which is why you never get it all right at once.

Lex asked Guido to define debugging. The answer came in layers. The shallow layer is the stupid mistake: a missing letter, an unmatched bracket, a wrong path. The middle layer is a logic error: the program does exactly what you wrote, and what you wrote is the wrong idea — a bad algorithm, a wrong invariant. The deep layer is spiritual, as Lex put it: the code is correct, the logic holds, it runs fast and stable, and it is still not useful. You built something that does what you told it to do, and that thing should not exist, or it makes the product unusable.

Guido added a fourth layer he wanted called out on its own. A program can seem to work really well 99 percent of the time and then do weird things 1 percent of the time on some edge case. “That’s pretty much all software nowadays,” he said — all *good* software. For bad software, that 99 goes down a lot. The uncomfortable implication is already in the room: every system you touch, including the ones under banks and hospitals, has paths nobody has walked. Maturity in this craft is learning to put guardrails on those paths rather than pretending they are not there.

It is not only complexity that forces the two-audience design. You are constantly improving the cherry-pie recipe, in a group or as last year’s mad scientist coming back to add icing. Style rules sound like a joke in most trades. In software they belong in the core documents of the language, because the recipe will be rewritten, handed off, and reread under pressure. Debugging, in Guido’s telling, is the name we give to that whole loop — typos, wrong thoughts, and building the wrong thing — plus the edge case that only shows up in production.

The first thing people notice about Python is its shape: a staircase of blocks stepping inward, like a carefully typeset business document. Guido’s explanation is plain. Space is how any text becomes readable. Strip a cookbook of bullets and crunch the sentences into one block, and in the kitchen you cannot tell ingredients from steps, or where one step ends. Indentation is nested grouping you can see: a block of text, then a smaller block inside it, then a smaller one still. Indent too far and after two levels there is no room left for words. Indent too little and you cannot tell a top-level bullet from a third-level one.

Four spaces is the compromise that fell out of old typesetting habits and the typical width of an 1980s screen. It is not a theorem. Google’s Python style guide — and, Guido thinks, Google’s C++ as well — uses two spaces per level, which makes the skeleton harder to see at a glance if you are unused to it. Classic Unix went the other way, a whole tab stop of eight spaces, which looks weird to him because three levels eat the line. In C, Rust, C++, and Java, indentation is a recommendation: the compiler will still accept a one-line mess, though colleagues and IDEs will yell. Python is different. You can indent two spaces or twelve if you want to go wild, but everything in the same block must line up. Indentation is part of the language, not a style suggestion.

Lex asked the historical question: was there a universe in which Python used braces? “Absolutely,” Guido said. For a long time he was convinced indentation was simply better. Strip away context and he still thinks so — it reduces clutter. Context, though, is almost everything, and most programmers know several languages. Apart from Python and maybe Fortran, Java, Rust, C, C++, JavaScript, TypeScript, and Perl all mark blocks with curly braces. Python is the odd one out. A Java programmer’s first day feels like walking and noticing you have been swinging your arms without thinking. Why not switch? “There’s no chance,” Guido said. Python already uses curly braces for something else, dictionaries mostly. You could not redefine C to use indentation either, even if you agreed a greenfield language should. Languages cannot move that kind of load-bearing wall.

Lex pushed the thought experiment to the end. If Guido could travel back thirty years, compatibility no object, and start Python over, would he still pick indentation? Yes, for three reasons. It frees a pair of matched brackets, “of which there are never enough in the world,” for other purposes — and in Python they later did other work. It is slightly easier for people who have never coded: a thirteen-week intro course already has to cover algorithms, data structures, syntax, variables, loops, functions, recursion, classes, expressions, and operators, and semicolons are the classic time sink. Python has no semicolons in that sense, so you cannot forget them or put them where they do not belong. The flip side, which Guido volunteered, is that forcing strictness teaches beginners that programming values attention to detail. You do not get to write the way you write English. They will still get that message from other details.

Lex switched targets to the dollar sign PHP puts in front of variables. In his head it does not parse, it is annoying to type, and it is easy to get wrong. Guido answered with archaeology. The mark comes from a lineage that includes PHP, Perl, and the Unix shell — and, before that, IBM job-control language. The earliest shells could run a script, a few lines of text, each line a command. A tiny processor took the first word as a program name and handed the rest of the line through as text. There were no parameters. When people wanted variables, especially arguments, they hit a collision: `program x y` already meant *x* is a file name. They needed a way to say *x* is a variable whose value is the file.

In the 1950s and 1960s the script interpreter itself had to be a very small program, small enough to fit in a thin slice of memory. The cheap trick: scan character by character, and if you see a dollar sign, jump to another piece of code, gobble until the next space, and treat that as a variable name. The same mark helps the human twenty lines down: `$xyz` is a parameter, not a path on disk. Those leftovers now sit behind Facebook-scale backends and, Guido noted, a lot of Wikipedia. Lex compared it to DNA, whose inner workings have been stable for hundreds of millions of years and are full of quirks you would never design into a self-replicating molecule from scratch.

Lex argued that biology has strange redundancy and still looks resilient at the organism level, while code does not. Guido disagreed. You would be surprised, he said, how much resilience modern code has. Even well-tested software that works in practice is full of things that do not work, absorbed by error-correcting layers — including the user. The user told to reboot the PC is part of the system. The milder version is reload the page, which everyone does a few times before admitting something is really weird. Lex said people should unlearn the extra click, because the second press often just turns the light back off. Guido’s lesson was drier: do it three times. Lex still finds the dollar sign “syntactic poison,” a bug generator posing as documentation. Guido held both sides: it *is* a kind of documentation, and it is also a source of bugs.

Lex wondered what a table of bugs per thousand lines would look like if you lined up nuclear-weapons code next to Twitter, Facebook, Dropbox, Google, and Windows. Guido said it depends on whether you have read the research. The last papers he remembered were from the 1980s and 1990s, across languages, companies, and styles, and the number was stable: about one bug per thousand lines in mature software that is considered as good as it gets. Lex then read a developer-side report from a programming-analytics company: on average a developer creates 70 bugs per thousand lines, 15 reach customers, fixing a bug takes 30 times longer than writing a line, developers spend 75 percent of their time debugging, and the United States burns $113 billion a year identifying and fixing bugs. Guido’s first reaction was that he had been off by an order of magnitude. His second was the distinction that matters. One per thousand describes software that has been tested and has settled. Seventy per thousand describes the moment you start typing. They do not measure the same thing.

He also read the page as marketing. There is a contact-us button at the bottom; spend a little less than $113 billion and the vendor will solve it for you. The silver bullet that makes bug-fixing investment go away is, he said, not going to happen. Lex tried to open a Stack Overflow report on the same topic. The page said the site was offline for maintenance. “It’s ironic,” Guido said. Then he explained the gap between 70 and 1. Software development is extremely iterative. Nobody really makes a complete plan for what will ship a year from now and works out every detail, because those details *are* a program, and a program’s plans have bugs and inaccuracies too. What you actually do is type, run, see whether it is what you meant, change it, and run again. After dozens of rounds, the residue is what “one per thousand” tries to describe. Code at birth is dirty. Engineering is the work of washing it.

Guido volunteered that he is a bad typist. He uses all ten fingers, badly; he never took a typing class. The first time he seriously learned a QWERTY layout was in college, on punch cards, pecking code with two fingers. Lex’s field method: watch anyone demo four lines and count the backspaces. Some people, especially with an audience, erase twenty, thirty, forty characters to fix a typo at the start of a line. Many who mistype a twenty-character word wipe the whole word and retype it three or four times. Mistyping a word and correcting it immediately is not really a bug, Guido said, though you lose time and occasionally a typo survives into the next ten lines. Forgetting to initialize a variable is a different class of error. In Python it usually shows up as soon as you run; modern compilers catch a lot of it too. The nasty case is the cousin: you initialized something, the name looks right, and it is the wrong object, or a colliding name. Compilers miss that. Tests may miss it. It waits for a night in production.

Lex uses a Kinesis keyboard because Backspace sits under the thumb. On a normal board the key lives under the pinky, and a lifetime of reaching is a kind of pain people stop noticing. Guido asked what happens on someone else’s standard keyboard. Lex’s answer was practical: you can spend a life carrying the keyboard with you. On a plane, if he expects to program, he opens the laptop and pulls a Kinesis out of the bag. “You have to accept who you are,” Guido said. Comfort setups are real — some people need a particular blanket to watch a movie; Lex needs that keyboard to reach focus. He tries to use state-of-the-art IDEs, but the place he can sit still is still Emacs. That tension, how tightly to hold comfortable tools versus how much to invest in new ones, is the argument the rest of a career keeps having with itself.

The signal from the community, Lex said, is noisy. Every year a crowd gets excited about new tools, and you have to guess whether they define a generation or will pass like weather. Front-end JavaScript is the loud case: frameworks, bundlers, and state libraries arrive and leave in waves. He had learned ActionScript in Flash, including graphic animation, and he had written a lot of Java applets, honestly thinking they might be the future of the web. They were not. Guido did not comfort him that the hours were free, and he did not call the work garbage. The particular technology gets replaced; many of the concepts it introduced, or first made accessible, are preserved. Java applets are gone. The idea of a page that contains little bits of code and responds to a button, a link, or a hover has not gone anywhere. Flash made animations painfully complicated; JavaScript equivalents were a better way to do the same job. The animations are still there. “There wouldn’t be jet planes without propeller planes,” he said.

From a user’s point of view that evolution is easy to bless. From a programmer’s point of view Lex wanted the harder ledger: nights in ActionScript, nights debugging applets. Java knowledge transferred. Flash did not. If you learned ActionScript early enough, Guido said, it was a super valuable skill *then*; the skill is no longer in demand. That is the calculation when you pick new languages and new IDEs. When you are young you have a whole life ahead, and you should feel encouraged to do a bit of stupid stuff — try not to get killed or maimed, but deviate. Nine times out of ten you learn why everyone else is not doing that. One time in ten you find something better. Once you are married with kids you get more risk-averse, because more is at stake and you have hopefully already had a window for crazy experiments. Early all-in choices — PHP, .NET, JavaScript — can look like picking a camp for life, but only as far as the technology stays relevant. If you learned C at sixteen and C were dead at twenty-six, there would still be time to switch.

Lex described his own hard leap. Most of his life he had programmed in C and C++, and with that hammer everything looked like a nail, including scripting. Arriving at Google, before TensorFlow was the obvious stack, he saw that C++ was not the right tool for machine learning — community and culture as much as language. Switching to Python felt like an existential crisis, like walking away from a person you love. C++ is still one of the most used languages on earth, still evolving, still a rabbit hole of metaprogramming if you let it pull you in. The question that is hard to ask while you already have a loved one is which language you are more productive in, and which one you enjoy your life with. He still writes WordPress backends in PHP and still runs the same triangle in his head: Node because the front end is JavaScript, Python because so much of his non-web work is Python, PHP because most of the web still runs on it and experience is already paid for. There is not a single right answer. It depends on time, stage of life, current work, and which communities you like.

Looking back twenty years you can call the ActionScript detour a waste. Nobody could have known. Guido’s advice was to accept that not every choice will be perfect, keep a Plan B, and not build a spreadsheet that estimates lifetime millions per language the way you pick a stock. Lex pushed back: if you carefully analyze where you can have impact, some language choices really are better, and they are about community and trajectory, not syntax. Guido’s objection was the inputs. “Better” is one of the hardest words in the language; better for you is not better for someone else. Community quality is hard to measure after the fact and harder to predict. You can build a gut. Your gut can also be wrong. That is why there are billions of people. There is more room in the Bay Area for people who plan to build a Google-sized company than there is room in the world for Google-sized companies, and they will have to duke it out — and the programming language is only one of the choices.

The fisherman, Guido joked, had long since jumped off the boat. The programmer got CPython back. The project planned to keep going 3.8, 3.9, 3.10, 3.11, all the way to 3.99 — and after that, probably 3.100. Python 3.11’s sexy claim was speed. How, beyond hiring a great team, do you make a thirty-year-old interpreter faster? Guido went back to how he started. He had not expected fame. He wanted something working and useful in about three months, so he cut corners, borrowed ideas in both language design and implementation, and wrote much of the code as simply as it could be. Many things can be coded more efficiently by adding more code: a time-space tradeoff. The simple version recomputes from scratch every time, looks obviously correct, and is easy for other programmers to follow. As requirements and performance pressure rise, you rewrite with more memory, remembering previous results. His classroom example was primes: the naïve loop asks whether *n* is divisible by every integer up to *n* minus one; a slightly wiser loop stops at the square root, skips evens after two, and finally checks only previous primes. Before you beat that, you need several PhDs in discrete math. The same story applies to string matching. Simpler algorithms are the ones people invent first. When you are designing a language you have hundreds of little problems, and you need a solution for every one of them before you can say you have a language.

CPython is an interpreter, a recipe for understanding recipes: given the text of a program, how do we run it. Python is always called interpreted, but there is a compiler in there. It does not compile to machine code. It compiles to bytecode for an imaginary machine — the Python interpreter. For 3.11 they tweaked only minor bits of that compiler. They did not build a just-in-time compiler, the Java-world trick now applied to almost every interpreted language. They made the interpreter a little more efficient. The key idea is an adaptive specializing interpreter. Bytecode looks like machine instructions for an imaginary CPU that can call a function, add two numbers, print a value. In Python there is no such thing, at compile time, as adding two numbers. The compiler does not know you are adding integers. You might be adding strings, lists, or two instances of a user-defined class that implements `+`. The plus sign is a user-interface trick: a category of functions written with one symbol.

The old `ADD` bytecode was pointers all the way down. Look up an object’s type, ask whether that type defines add, find a function pointer, call it; the function then checks the other argument’s type and does a pile of error handling before it can add the bit patterns. Integers, floats, and strings each interpret the same bits differently. The optimization is local to a line of code. If you call a function a thousand times, that plus is likely always integers, or always strings. So each time an add runs, the interpreter keeps a little note of the type that actually did the work. After a few identical hits it rewrites the generic add into an add-integer operation that assumes both sides are integers and does the work inline. The “lie,” Guido said, is that Python can still hit that same line later with floats, strings, or a mixed pair. The specialized op still checks that both arguments are integers, with tricks to make the check cheap, and falls back to the generic add if the guess is wrong. You waste a few cycles on a miss. Statistically you hope to guess right, because guessing wrong too often can make things a little slower. Someone could construct a counterexample that runs five times slower in 3.11 than in 3.10. It would be an extreme fluke, and a fun reverse-engineering task.

The heuristic is almost embarrassingly simple: assume tomorrow’s weather is the same as today’s. That already beats guessing at random. Python is not the first language to do this; the trick is well known from other interpreted, dynamically typed languages that needed to get faster, including papers about Facebook’s HHVM for PHP and ideas from the JVM and academia. It matters *because* Python is dynamic. If the compiler could read that `x` and `y` are integers, it would emit a single hardware add. In Python you generally do not declare types, or even declare that variables exist — they spring into existence on first assignment, which is kind to beginners and expensive for the interpreter. The challenge of 3.11, Guido said, was always the same challenge: make interpretation more efficient without losing the super-dynamic nature of the language.

Python 3.5 got PEP 484 type hints. Lex asked what they are, and whether the interpreter uses them. They are optional, Guido said, and especially popular at larger companies with very large Python codebases. They are more than documentation: a sub-language in which you can say this variable is an integer, this argument is a string, this function returns a list of strings. They are not checked when you run the code. A separate static type checker reads the source without executing it, thinks hard about what the code appears to be doing, and double-checks that against the annotations. It is a development tool, like a linter. The annotations are not used to speed up the interpreter, for several reasons. Many people do not use them. When they do, they sometimes contain lies: the checker cannot prove an integer is ever not an integer, and at runtime someone still violates the assumption, and the interpreter is fine. If Python started enforcing annotations, many programs would stop working, and some programs would not even be possible because they are too dynamic. There is a possible future, three or four or five releases out, in which annotations become hints for an add-integer instruction with the same generic fallback. They are not there yet.

Third-party libraries can already take a hint and enforce it at runtime, almost like a manual assert. Python is unusually introspectable: while a program runs you can ask a variable its type, ask a function its arguments, and now ask its type annotations. The cost is the opposite of 3.11’s project. Extra runtime checking slows you down. Given that sales-pitch claim that developers spend 75 percent of their time debugging, Guido would sometimes pay performance to catch type errors — and in most cases, doing it statically before production is more efficient than doing it piecemeal at runtime. Mypy is the original static checker. It was started by a Finnish developer, Jukka, who Guido thinks was living in England when he invented it. The PEP 484 syntax was developed with that checker. Jukka had first used a different syntax, angle brackets in the C++/Java-generics style, which would have been too tricky in Python. They already had a dummy syntax for annotations and no agreed meaning for it; types were the obvious use. There was still no direct syntactic way to write generics in the language, so early mypy was literally “my Python”: a preprocessor that type-checked, then stripped the extra marks. Preprocessors sit badly in typical Python workflows. If that model had won, Guido noted, it could have been a TypeScript-style split.

Asked whether he would recommend TypeScript or JavaScript, Guido picked TypeScript for the strictness: it keeps your head straight about what the code is doing, helps editing, and remains compatible. A pure JavaScript library can be used from TypeScript and the other way around; that compatibility is the key to TypeScript’s success. In the JavaScript world transpilers are already the standard workflow, so TypeScript being a transpiler is not a big deal. A stage-one proposal would have JavaScript ignore a superset of TypeScript’s annotation syntax at runtime, the way Python ignores hints — meaning you could someday shove TypeScript into a JS engine without a transpile step. Browsers now update their engines fast enough that translating JavaScript itself matters less. Lex feels JavaScript “eats the cake” on bugs per line compared with Python, a beautiful mess under intense evolution. Guido said culture differences are partly random and partly environment: JavaScript is essentially the only language for client-side web apps, deployed across every shape of screen, and that produces a different community than Python’s longer history of large, stable systems.

Static typing in Python is still controversial, Guido said, but much more accepted than when mypy and PEP 484 were young. Mypy is written in Python, and most later typing features were prototyped there. It succeeded enough that companies decided the idea was worth investing in — and then, instead of funding mypy itself, Google, Facebook, and later Microsoft built their own checkers. Facebook’s Pyre, written in OCaml by people who had shipped HHVM, fit Facebook’s workflow. Google’s pytype, written in Python, was built for a gigantic monorepo. Microsoft’s horse is Pyright. A type checker, philosophically, lives outside the language, like a focused linter: compilers will not tell you an unused variable is unused; a linter will, along with PEP 8 naming and a pile of heuristics. Nobody is currently excited about integrating a checker into Python itself. Checkers still ship monthly; Python ships once a year, and syntax, once added, takes many releases to remove. Guido would not put a number on adoption, but anyone with a CI cycle probably runs a checker on every commit, and in most cases that checker is still mypy. A page Lex found claimed 20 to 30 percent of Python 3 codebases use type hints, apparently from sampling GitHub.

Lex asked the biased question, now that Guido was at Microsoft: best IDE for Python — PyCharm, VS Code, Vim, or Emacs? Historically Guido started with vi, then spent from the early 1980s until about two years before the taping in Emacs. Between roughly 2013 and 2018 he dabbled in PyCharm, mostly for a few features he could not get from his Emacs setup. PyCharm, he said, is like driving an eighteen-wheeler. Emacs is the comfortable Toyota you have had for a hundred thousand miles, every rattle familiar. He was very comfortable in Emacs, but it was not very good, at least as he had configured it, at finding the definition of a function.

At Dropbox he was exploring a five-million-line Python codebase. Grep for a class `Fubar` and you find many classes with the same name. Once PyCharm had indexed the repository it was very helpful for that kind of navigation. As soon as he had to *edit*, he jumped back to Emacs, where he could type faster and switch files much more quickly when he already knew which file he wanted. He never really got used to the PyCharm UI. Lex confessed the same split and the same guilt: feeling like a grumpy old man for not learning the shortcuts, the way refusing to learn ten-finger typing limits you in the long run if you do a lot of keyboard input. He had even switched from physical books to Kindle to let his brain get used to a digital future. PyCharm, he said, is the most sophisticated, featureful Python IDE; at some point he felt he should be forbidden from using anything else, or VS Code, and walk away from Emacs for this job.

Guido would not make that choice for anyone else. He gave VS Code a tentative try when he started at Microsoft and really liked it, then took a while to understand why. The VS Code founders may not agree, but to him VS Code is the spiritual successor of Emacs. The heart of Emacs is that it is mostly written in Lisp; new features are new Lisp packages, and you upgrade for those, not for an obscure change in the C core. A small engine reads files, puts bits on the screen, and manages buffers; almost everything in that core can still be overridden. VS Code has a similar architecture: a base engine almost nobody but its owners changes, a package manager, interfaces and conventions for how packages talk to the engine and to each other, and primitive operations for cursors, selections, and input. The extensibility and the package ecosystem mirror Emacs. JetBrains IDEs — PyCharm, PHPStorm — are specializations of IntelliJ, one editing engine with a different veneer, and third-party extensions are possible but a struggle, not part of the culture. Years earlier, trying to get better mypy integration into PyCharm, the engineer writing the plugin could barely find documentation or a debug workflow. It was not a pleasant experience.

Lex had read Guido’s post “Reasoning about asyncio.Semaphore,” which opens with a Silicon Valley fast-food restaurant that has only one table. Was it real? A metaphor, Guido said, one of his rare literary moments, a crazy example to catch attention before dry material about locks and semaphores. (Lex had looked it up: restaurants like that exist, often in Tokyo, one table, a line, a meal cooked just for you.) For the fisherman now back on the boat, concurrency is two fishing rods. Fishing, in the sit-and-wait style, is mostly waiting for a nibble; one person can run three or four rods and catch more fish in the same Saturday. In computers the words split. Parallelism, when people say parallel computer, usually means several complete CPUs tied together, sharing memory or an I/O bus. Concurrency can be more abstract: the illusion that things happen at once, while the machine actually runs this program for a while, then that one, then a third. Parallelism is the reality of extra hardware. Concurrency is part reality, part illusion.

Implementing synchronization primitives is hard, Guido wrote, because at the conscious level our brains are not trained to track multiple things at once. You can walk and chew gum; try balancing a checkbook while watching a murder mystery and you will mix up the digits or miss a clue. The programmer, with the current state of the art, is still responsible for writing the code correctly. It is hard enough to keep a sequential recipe in your head: what is in `a`, what is in `b`, what is in `c`. If one piece of code uses `a` for fishermen and another uses the same `a` for mermaids, sequential execution can finish one use before the other starts. Mix them and the mermaid count overwrites the fishermen. You have to anticipate orderings. You think you are smart and you put a lock around it. In bugs per thousand lines, this is an area where everything is worse. A lock is the sign on a single oven: in use, or free. A semaphore is the electronic board that says five ovens are still available, or three people are already waiting — so an idle oven may already be reserved for someone earlier in line.

Asyncio landed in the standard library around 3.4, after an old pair of modules from the late 1990s or early 2000s had stagnated. Around 2012 or 2014 someone asked for one tiny feature on those modules, everyone said you were not supposed to use that stuff, and a fight broke out over which third-party library was the right one. Guido took that as a cue for a better standard-library module. The PEP was, at the time, the largest ever proposed; he was lead architect, borrowed from existing libraries, and put the result in the standard library. Type annotations later distracted him; other people kept asyncio alive, and it has been successful, especially for Python web clients. Two architectural camps exist, in operating systems as well as in Python: Windows prefers one style, Unix the other. One is evented — notify me when a packet arrives. The other is a dedicated thread of control, blocked on one connection. Several traditional Python async libraries used callbacks, stacks of paper you sort incoming sheets onto, a style of spaghetti Guido found aesthetically unpleasing and that was prevalent in JavaScript’s event loop. He chose the task-based model, reused language mechanisms Python already had, and built the library around that paradigm. It turned out to be a pretty good one.

The global interpreter lock, the infamous GIL, solves a problem from the beginning: Python was not written with asynchrony or parallelism in mind. There was no concurrency in the language, no threads. A few years in, SunOS, IRIX, POSIX, and Windows all grew threading libraries. The OS would handle threads; the program could pretend it had as many CPUs as threads, and if it did not, the OS would simulate them. Python was not written for that. Every vendor was adding a slightly different C threading API. In the early 1990s threads looked like a cool new toy, and one of Python’s niceties was giving programmers a *safe* version of new operating-system toys. Sockets had been a literal translation of BSD sockets that turned C’s silent hangs into clear Python errors. The team wanted the same for threads, and did not want to rewrite the interpreter to be thread-safe. Every object assumed there was only one thread. So they took their losses: something that looked like threads, and on a single-CPU machine — which most computers then were — felt like threads, because even one CPU could run ten or twelve of them. Then chip designers stopped making cores faster, started putting several on one die, and the GIL became infamous. All those Python threads still ran on a single core.

Could it go away? The most likely future Guido saw was multiple sub-interpreters, each running a completely independent Python program, with some faster communication between them that the runtime would manage. You would still spend more time talking between programs. Shared objects are the attraction of threads and also their downfall: you share what you did not mean to share, and concurrency bugs follow. That is why he wrote the semaphore post. A no-GIL, free-threaded Python would also cause a lot more software bugs. There was already a possible experiment: a Facebook engineer had forked CPython into a “no-GIL” interpreter, with enough optimizations that the single-threaded case did not run much slower and the multi-threaded case could use all the cores. The question for core developers was whether they would maintain that complexity indefinitely, and whether they would accept overhead on the single-threaded path. Guido was not convinced enough people needed multi-thread speed in Python to take that hit. The GIL, for him, is a pretty nice Goldilocks point between no threads and all threads all the time. Not everybody agrees. Sub-interpreters looked like a fairly safe bet for 3.12, about a year out, on the yearly release cadence.

Will there ever be a Python 4.0, given the pain, joy, suffering, and triumph of the 2-to-3 move? The community and the core team have painful memories of that transition, which is one reason everyone is happy there will not be a 4.0 anytime soon. If there is one, they would plan the transition very differently. They underestimated what users could absorb. Had they known, they could have designed Python 3 somewhat differently without making it worse. Python 2’s official end of life on January 1, 2020, was a symbolic moment for remaining users; the team had moved on years before. They had planned a party at that year’s U.S. Python conference, which the pandemic cancelled. Guido’s personal sketch of a 4.0 is not new syntax. If they ever shipped a no-GIL Python, it would feel different enough for the part of the ecosystem built on C extensions — which is the entire machine-learning, data-science, scientific world. Suppose after 3.19 they named a release 4.0 and flipped the switch. He would want that year to add no new syntax, no new standard-library modules, no new builtins. At the Python level it would be compatible with 3.19. Extension modules would all have to be recompiled. The binary interface would change. Pure-Python users would have a breeze, except there are very few left: everybody doing something significant uses third-party extensions, hundreds of thousands of them on PyPI, some already low on maintainers.

The humane path would be years of warning. Maybe at 3.15, no-GIL becomes a compile-time option: change one configuration variable, rebuild, and also rebuild every extension you need. That is impractical until those extensions exist, but it lets maintainers test the waters, find APIs that will vanish, and write alternatives. A series of releases would grow solid no-GIL support, and 4.0 would be the mayor cutting the ribbon — no-GIL the default, maybe the only mode. Asked for must-have PyPI libraries, Guido had no canned list. When he writes Python it is usually tooling around building Python itself, and the last thing they want is third-party dependencies, so he lives in the standard library. Missing pieces nowadays are usually new ideas with one or more third-party implementations evolving faster than the standard library could. Incorporating them would slow them down. He likes a lively package ecosystem, and recent standard-library work has included spring cleaning: modules he spiked in 1992 or 1993 whose commit history is cosmetic, APIs identical to twenty years ago, sometimes better off replaced by a third-party alternative.

Does it make sense to Guido that Python became the dominant language for machine learning — PyTorch, TensorFlow, scikit-learn, and underneath them NumPy, SciPy, pandas, matplotlib? Part of it, he said, is as simple as everyone driving on the right side of the road: compatibility. They could have picked JavaScript or Perl. In the early 2000s it really looked as if Perl would dominate the biosciences, because DNA search is regular expressions and Perl still has the fastest, most comprehensive engine. Anything was better than writing the scientist-facing layer in C or C++. A man who had worked in the computing division at Lawrence Livermore sent Guido his memoirs of helping “computational steering” into existence: take libraries, in his day written in Fortran, that solve universal mathematical problems, and let scientists who keep asking new questions tie those libraries together in a higher-level language. Paul DuBois was one of the people who, in the mid-1990s, saw that need. When Guido first created Python he was not expecting it to be used for arrays of numbers. He thought that was an outdated data type; Python was good at strings and objects, and multidimensional arrays did not even exist in the language. Other people realized the extensibility was flexible enough to write third-party packages that did large arrays efficiently.

They got a foothold in different parts of the scientific community. Guido remembers Hubble Space Telescope people in Baltimore as big Python fans in the late 1990s. Once scientists working on similar data — different questions, same underlying libraries — can exchange a Python library instead of starting from scratch in Perl or JavaScript, the lingua franca feeds itself. New fundamental libraries then pick Python because their users already know it. That, he imagines, is how TensorFlow ended up with Python as the user interface. Lex added the competitive history: Google’s internal predecessor, Theano, Caffe, some Scala, but Python already dominating. MATLAB had been the engineering alternative, and MathWorks’s choice not to make it open source, plus a price per seat universities disliked, mattered. Guido thought universities would still pay. The deeper miss was that MATLAB did not feed GitHub’s open-source package culture, the tinkerer culture that lets you build a package from scratch, get excited, and share it. Python’s open source, as he likes it, is almost egalitarian. You still have to decide whether to drive on the left or the right, but you do not have to work for a big tech company to make a difference. The PSF spends most of its money on community-fostering events, not on development.

Four years earlier, just after he had stepped down as benevolent dictator for life, they had talked about the role. Looking back, he probably held on too long. He remembers being extremely stressed for a long time without a clear picture of what was causing it, and he thinks he should have relinquished the central role sooner. The benefit for the community was clarity of vision. He had let himself be influenced as Python grew, and he had also stuck to certain principles — still hard to name as a list, but the fact of the role modeled how to think about problems. People learned how he thought and could predict how he would decide. There was not a lot of jerking around, Democrats-in-power then Republicans-rolling-it-back. The successor steering council has found a similar way of leading without stagnating. For him personally it is more fun. A bug in multiprocessing? Let someone else decide if it matters. He can stick to typing, asyncio, and the faster interpreter.

Culture differences among the companies he has worked at are real. Dropbox is much smaller, a narrower product, a smaller codebase, and during his time it had a habit of putting the whole company behind a one-year plan, then discovering everything was wrong, plus an annual engineering reorg whenever a new VP arrived. Lex still loves the product: effortless sync, conflicts handled, the kind of service you are not supposed to think about. Guido said there was a lot of internal hand-wringing, and the product always worked smoothly. The lesson he would draw is familiar: if the product is good, keep doing the thing you do extremely well. Microsoft’s shift under Satya Nadella — buying GitHub, embracing open source and developers — is why Guido joined after a brief retirement the pandemic made him realize was a ridiculous thought. Work can be a source of fulfillment. He has been talking to Excel people; the spreadsheet is about 35 years old and can still read a file from a floppy. Nadella, whom he has never met, he hears is a sharp thinker with serious business sense, who turned a company that had been openly hostile to open source into one that actively embraces it. Excel will not suddenly go open source. There is room for a product like VS Code, which is.

Guido had written a blog post answering someone with a CS degree who was choosing between a nine-to-five job and becoming an entrepreneur. Creating Python was, Lex noted, an entrepreneurial act, and good programmers keep facing the same fork. Guido’s map was wider. You can create something new inside a big company; if it grows, you can rise to run it, if that is your aspiration. You can also be the brilliant engineer who builds a first version and has no wish to manage a team from five people to a thousand, then go invent the next crazy thing, or found a startup, or move. Drew Houston started as technical inventor and was still Dropbox’s CEO, still present, when Guido was there — someone who had the skill set to grow and stay on top. Other people are brilliant engineers and horrible at management. Guido counts himself in the second category. His first love is still to be an individual contributor, the programmer.

Advice for a beginner on how to learn Python the right way: find something you actually want to do with it. “I want to learn skill X” is not enough motivation. Pick a problem, even a crazy unrealistic one, that challenges you into actually learning to code. It does not have to be ambitious: a small social-media bot, a Twitter bot, automating something you do every day, processing files. Machine learning is a sneaky on-ramp. An hour-long tutorial can get you using pretrained models to do something cool, and along the way you learn just enough imports, I/O, and function calls to fall in love first and learn the basics later. You cannot expect to learn Python from a one-hour video. Guido blanked on the name of someone who wrote a funny post against ads for “Learn Python in 10 days”; the gag was that the goal should be learn Python in 10 years. He disagrees with the slogan and agrees with the criticism: five-minute tutorials are usually bad. You can learn a hook in an hour. Memorizing syntax does not make you a coder. The ten years is the experience and the pain along the way.

Coding has already changed: so much of it is copying from Stack Overflow and adjusting. Guido did not want to talk that style down; it is efficiently. Then there is the next step. He uses GitHub Copilot every day. It writes a lot of code for him, usually slightly wrong, and still saves him typing — change one word in a line it generated perfectly. Sometimes, if he names a function right, it writes a five- or ten-line body, and he knows Python well enough to spot the issues quickly. It becomes a dance partner. It does not save him a lot of thinking. Because he is a poor typist, he is grateful for the keystrokes, more than for the older dot-completion menus still built into VS Code. Are programmers’ jobs threatened? “Are programmers’ jobs threatened by the existence of Stack Overflow? I don’t think so.” It takes care of the boring stuff. You should not use it to do something you have no way of understanding. It is best when the question is *please remind me how I do this*: whether the method is `foo` or `bar`, builder or constructor or factory, what the parameters are. A great assistant. The creative work of deciding what you want the code to do is totally yours.

What is the future of Python in ten, twenty, fifty, a hundred years — metaverse, Mars, humanoid robots? It will eventually become a legacy language that plays an important role, that most people have never heard of and do not need to know about, “just like all kinds of basic structures in biology, like mitochondria.” It permeates digital life; people build on top of it and only know the upper layers. Most programmers rarely need binary arithmetic. Guido started as a teenager building little digital circuits from NAND gates he made with transistors and resistors, and he feels blessed for that start. He can walk a long way down toward shift registers; how you actually make a chip from silicon is magic to him. He covers a middle layer of the stack, loses the plot below NAND gates and at the top when it gets to machine learning, which may build higher layers that help us understand physics — or may not. Human brains do enormous parallel work, like recognizing a face, then use that circuitry to simulate a single-threaded, much higher-level interpreter: the illusion of one thought after another. DNA is a seed you plant, not a computer you buy. Cells replicate, organisms replicate, societies self-preserve, and from an alien vantage the interesting organism might be life on Earth, or civilization, not the individual staring at this particular layer.