A Hundred Million to Train, a Penny to Infer

Nathan Labenz is sharing a special cut of the Moment of Zen podcast, which Erik Torenberg co-hosts with Antonio Garcia Martinez, founder of the web3 growth-metrics company Spindle, and Dan Romero, founder of Farcaster. The guests are Amjad Masad, CEO of Replit and a seller of AI coding assistants through Ghostwriter, Flo Crivello, founder of Teamflow, and Labenz himself. They trimmed an original hour-and-a-half conversation to about thirty minutes of AI argument: visionary takes, skeptical takes, and a fight over whether this moment is different from previous hype cycles.

The preview already gives away the clash. One side describes exponential improvement so tight that you could go to lunch and come back to a changed world, then follows that to the conclusion that the most likely outcome is the death of humanity, because you cannot align an enormous computing force that does not understand human preferences. Antonio's reply is a single line: he thinks they are completely full of it. The rest of the table is operators trying to decide, in real time, whether they are watching AGI, a new kind of computer, or a nerd fantasy with a fundraising deck.

Flo frames the moment as a third great convergence. An AI researcher had described the first as 2012 and the ImageNet deep-learning turn, when the whole field piled into neural nets; the second as Transformers and the 2017 paper *Attention Is All You Need*; and the third as large language models from roughly 2022. Those models are extraordinarily good at manipulating language, and it turns out many tasks can be modeled as language tasks: questions, a bit of math, search, code, even a reasoning engine for robotics. Flo thinks AI is still dramatically underhyped. Even if discoveries froze today, he says civilization would be dramatically impacted within ten years. His AGI timelines are compressing, and his concerns are slowly rising.

Amjad steels the argument for the table. If you accept the secular view that humans are meat computers, no physical law says you cannot build those computers in Turing machines, which means human-level AI arrives, which means an AI explosion: each generation trains the next. People at OpenAI, he says, are already using GPT-4 to train GPT-5; the loop is happening on a years-long clock and will someday compress to milliseconds. That is the singularity: you go to lunch and the world has changed, because you cannot know what is next. The Yudkowsky branch then says the most likely outcome is human extinction. An enormous computing force that is still "dumb" about human preferences cannot be given a goal specific enough; there are too many ways to interpret it that get you in trouble. Moore's Law implies there will not be one AGI but many — in the limit, anyone with a laptop has one — so you must do an impossible alignment job a million times. Flo points to Eliezer Yudkowsky and the LessWrong offshoot around Paul Christiano as the source of the view, and wishes there were far more funding and attention on the risk.

Antonio calls it nonsense. He hears in AI-doom literature the nerd fantasy that when the world ends, law and order collapse, and it is Mad Max, the people who keep the computers running somehow end up in charge. The word robot, he notes, comes from a play in which robots take over; Jewish folklore has the Golem, a created thing that rebels. There has always been a latent Western story that the artifact turns on its maker. Transhumanism, in his view, is the same story in lab coat: take a Kurzweil spiel, globally replace "singularity" with "Rapture," and you have an evangelical sermon. The speakers do not notice, he says, because they do not actually read religion. He is more open to an H.G. Wells *Time Machine* ending in which humans become Eloi on the surface while machines play Morlocks — and he thinks that has already happened in the sense that people get worked up about Twitter fights in which nothing happens.

The deeper error, Antonio says, is ascribing agency. Anyone with kids knows children name toys and even boxes and give them personalities; humans do that to models the moment they glimpse a hint of agency. He does not think today's systems formulate abstract goals, form desires, and go execute them. Large language models are a new kind of computer, a powerful tool. Recruiting enormous value in the world takes planning, emotion, and theory of mind — the work of convincing other people that entrepreneurs actually do — and he is not sure that can be built yet. Asked about AI girlfriends, he says they might look like it, act like it, and fool some people; ChatGPT already fools some people. That does not make them the thing, and it does not get them to the level of power that kills everyone.

What the Transformer actually brought, Amjad argues, is a jump in generality. Take today's large language models back to the early era of AI and those researchers would probably have called them some kind of AGI. Generality is a boiling-frog phenomenon: it rises slowly enough that there will never be a moment when everyone shouts that this is AGI. The rising tide makes everyone more productive and makes software much easier to create. Creating software used to mean learning arcane knowledge; now you write English and can produce something meaningful. Expert software-engineer capability is being handed to everyone in the world. The impact, he thinks, will be hugely deflationary and will give people superpowers. A new kind of entrepreneurship is appearing: people who once needed companies and armies around them can now build useful things on their own, and a new crop of millionaires and billionaires will come out of it.

Would he teach his kids to code? Yes, because generated code will not be 100 percent accurate, so you still need to debug. Surprisingly, most programmers already spend most of their time reading and understanding code, not writing it, and that tilt will get more extreme. The pressure lands on tooling: debugging, comprehension, visualization, and letting a model explain the code. On the front end, he expects super-productive people powered by generation who may not write much code at all. They will plumb things together, talk to a lot of different models, and act more like project managers and product managers than like traditional programmers.

Flo calls this the Steve Jobs black pill. The early vision of computing was that everyone would be a programmer; there was no user/programmer split. Jobs popularized the end user with lovely interfaces, and most people became consumers of software rather than creators. Flo thinks software creation will come back: personal software, tools for a business use case, throwaway code that does not need perfect accuracy or high performance. Professionals may not have to learn to code; people who want to be low-level programmers still will. The 10x engineer becomes a 100x engineer because you still have to guide the tool. He prefers a rising-tide image: the 1x engineer becomes 10x, and people who were never engineers start on simple tasks and then more complicated ones. It is not mainly a replacement dynamic. He also flags Anthropic's Constitutional AI paper as the next generation of RLHF — reinforcement learning from AI feedback rather than from humans.

Amjad names two hard weaknesses. Models are bad at systematic thinking: give GPT-3 enough examples of converting dates from day-month-year to month-day-year and it succeeds something like 99.8 percent of the time and fails 0.2 percent of the time. It kind of understands the task; it does not think about it systematically. They also cost a fortune to run. It will always be more expensive to parse a date with a large language model than to run a piece of code. He expects AIs to learn to use tools: if a task is frequent, currently failing, and benefits from systematic thinking, the model should write code to do it because that is cheaper, more reliable, and more systematic. Engineering, he says, is lagging way behind capability; everyone is scratching the surface. It is possible today to give ChatGPT a Python interpreter and a search engine and ask it to book a flight — search, write a program, hit an API — and nobody has really built that. Replit's Ghostwriter is one attempt to close the gap.

Amjad says he has long been a *Sovereign Individual* believer. That book described a future in which people are less full-time employed, more freelance, able to construct companies on the fly and dissolve them when the work is done. He thinks the technologies are finally maturing enough to support that picture. When he talks to younger programmers, almost without fail their ambition is no longer to join Microsoft or Facebook. They want to build businesses, freelance, and design careers that maximize freedom. An army of AI assistants, he argues, will give those people extraordinary leverage: one person becomes a team.

Antonio agrees and analogizes to how the PC empowered individuals. WhatsApp, he notes, sold for something like $20 billion with about 40 or 50 people. He thinks we will see a billion-, ten-billion-, maybe one-trillion-dollar company in a decade or so with perhaps one or two people. Dan Romero cuts in on compensation: if you really believe in 10x engineers, you should see them making $10 million a year at big companies, and you do not. Conversely you see people he calls professional coffee drinkers, who rest-and-vest and still take home a $500k package. Pay for output, and the rest-and-vest crowd shrinks.

Antonio is sure bounties will not be the mechanism. The bounty system, he says, is one of the parts of web3 that demonstrably does not work; companies built around paying a bounty to do a thing are the ones you have to route around. A 10x engineer who wants a $100 million exit will not sit there doing Mechanical Turk for coding all day. Most people who make real wealth in software are neither rest-and-vesters nor bounty chipmunks; they work hard on a committed product with a design ethos. Google needs 120,000 employees, most of them "making lattes every day," but at the drop of a hat the company needs them with very little friction. Hoarding talent is rational, like a billionaire keeping assistants who look idle until something important happens and the market cannot supply that labor instantly.

The pattern he offers is that whenever technology lowers the cost of going to the market, people go to the market more. Before Uber, more people kept personal drivers; before DoorDash, some people kept servants waiting all day for one order. Bounty-type systems and crypto might still help solve coordination and transaction-cost problems so you can have hundreds of coders working for you while you focus on a billion-dollar idea. Amjad adds an ethical-commercial wrinkle: if you could trace Wikipedia contributions token by token into a model, a company that wanted to do the right thing would assign value per token and pay a revenue share to the authors of the training data.

Labenz asks the practical question: for people who want to join or invest in AI companies, which kinds will endure and which will be commodities? Antonio slices the field into three: companies that train the big models, horizontal applications on top, and vertical applications. The moat for large language models is economies of scale. GPT-4, he thinks, is on the order of $100 million to train, between compute and the rest, which is why OpenAI is raising so much money, plus the cost of labels and RLHF. Training is a one-time cost. Inference — the cost to run the model once it is trained — is many orders of magnitude lower, more on the order of one cent.

He then knocks down the idea of an algorithm moat. Google is good not because its AI is so amazing, but because the entire world is typing what it wants: it was always the data set, never the algorithm. Algorithms are not particularly defensible. AI is not a product any more than linear regression is; you have to apply it to something someone will pay for. That is where he does not quite get the hype. If Google cannot find a way to turn this into cash, and a lot of companies are thinly veiled prompt UIs on ChatGPT, what is the actual product?

Flo answers with economics. Production of text, music, and images has not been where the value lies; you could already hire people on Fiverr to do those things. Whenever you make something cheaper, its complements become more valuable. Generative AI makes content cheaper, so the complement of content — distribution — becomes more valuable. He thinks that is great for TikTok, YouTube, and similar platforms. Production was never the scarce resource. Distribution is.

Flo's twenty-year claim is blunt: all knowledge work will be automated, coding first. In the next two years he expects a wave of action-oriented knowledge-work automation that actually does things, with customer support a large piece. A knowledge worker, as he puts it, is a function that sits between a keyboard and a monitor, and AI is going to be really good at approximating that function. Labenz pushes back: people have been saying this for twenty years; Marvin Minsky was saying it in the 1960s and 1970s. He wants to know whether this is a real inflection or the same trend line that already took the industry from Kernighan and Ritchie to Replit — a big deal, in his view, but not AGI or end-of-the-world levels of change.

Flo's distinction is that in the past, when AI could not do a task, it simply could not do the task. Now, across very broad sets of tasks, the best AIs are outperforming the average human. They are not yet at expert level in a specific domain, but they can outperform employed college graduates on a wide distribution of tasks. Labenz cites BIG-bench, where Google compared PaLM to a pool of roughly software QA testers and the AI won. Antonio says he heard the same discourse when Deep Blue defeated Kasparov. He quotes Edsger Dijkstra: asking whether a computer can think is like asking whether a submarine can swim. The point is that it does forty knots and the human does one. Ranking ads or filtering Stack Overflow, the computer already does better; that is different from saying it is more intelligent, and it still causes a crisis in the knowledge economy.

Amjad returns to generality. For the first time, models can learn your intent from one example: he can build "Google Translate Antonio" in three seconds by saying "English hello, French salut," then "English home," and it continues in French. The last jump in generality like that was the Turing machine. Every app will get better. The biggest beneficiaries, he thinks, will be growth-stage startups because they already have distribution: Notion adding AI is much more interesting than a new AI-first knowledge-management company. He is bearish on AI-first startups and bullish on AI infrastructure — OpenAI, prompt IDEs, fine-tuning, anything that makes it easier to build with Transformers — a diffuse technology every company will buy, similar to cloud.

On the financial question, he would run another FAANG-type public-market strategy because he thinks there is another S-curve, plus infrastructure names such as Nvidia. On the startup side, the beneficiaries are developer tools, the AI intelligence layer, and early-growth companies like Replit or Notion. The table never settles the extinction argument. What it does settle, for operators, is narrower: do not confuse a prompt wrapper with a product, put AI into products that already have users, and remember Dijkstra's submarine. The thing does forty knots. You do one.