Replit's Amjad Masad on Building Your Own Model, Latency, and the Next Billion Software Creators

Asked what he would tell someone starting to code today, Masad says his advice has not changed: you learn to code by making things. Most people learn by pursuing a goal and picking up skills along the way, which he calls almost the definition of intelligence. He contrasts that with an education system that teaches random facts, roughly 90% of which are never applied.

He notes that fewer than 0.5% of people have any exposure to coding, despite how powerful it is. His first job, as founding engineer at Codecademy, was built on the same learn-by-doing idea. LLMs push it further, because a Replit user can start from a template, ask the AI for a small edit, and see a result within minutes without wrestling with installation and setup.

Masad predicts that software work will split. Product creators, who may not care about code and only want customers, will spend most of their time prompting and iterating. Traditional engineers who build cloud infrastructure, data pipelines and backends will see less change. For the former, he says, you can just start building; for the latter, a computer science degree is still relevant.

Masad explains that Replit renamed its Ghost Rider AI product to simply Replit AI. He compares today's paid copilot add-ons to the early days of the GUI, when Windows was a $100 add-on to DOS. In his view this add-on era is transitional, and companies that depend on that revenue should be worried.

Replit instead embedded AI everywhere and put it in the free plan, so the first line of code a user types already gets a suggestion. There is a structural reason too: he wants designers to start from AI rather than compare a workflow with and without it. The product mixes a "push" model of passive code suggestions, a "pull" model where users generate whole files from a prompt, and small touches such as an AI debug button that opens a chat pre-filled with the console error.

Masad says that since the GPT-2 and GPT-3 era his view of LLMs has become more reductive: a model is a compression of its data, and its real power is interpolating between distributions, like writing a rap in the style of Shakespeare. That makes it a new programming paradigm, with data as the program. To judge what a model can do, he says, you have to understand what data went in and what post-training was applied.

On the coding front he offers a mildly bearish note: open-source tokens are running low, and GPT-4 was trained on all the code on the internet plus very expensive annotation by contractors. Replit's advantage is a large user base that generates unique tokens. He still expects two to three more years of improving capability, partly because adjacent data such as scientific and legal text has been shown to help code performance.

On what makes good data, he says size and diversity matter, and that high-quality data can be reused over multiple epochs. Training on minified JavaScript, for example, damages a model. Because GPTs behave like behavior-cloning machines, data from the best programmers yields better models. Replit has found that GitHub is full of high-quality infrastructure code but short on application code, which is where its users contribute.

Masad expected the return on learning to code to be highest for beginners, and he points to Replit users on a free 100 Days of Code course who went from nothing to building a company and earning hundreds or thousands of dollars within months. He cites a BCG consultant study in which weaker consultants benefited the most. Still, he suspects the advantage would shift toward advanced users if people were trained to use LLMs well, since they can debug and also seek answers across several models.

On user education, he says younger people simply rolled with AI suggestions when Replit made them free, while some established users and teachers were jarred. One teacher told the company it had destroyed their classroom. Masad's reply was that the feature can be switched off, but that AI is like the calculator and will be widely used anyway, and teachers who tried it found their students learning better within a few days.

Asked how to structure AI teams, he says he is an infrastructure person and has always favored a horizontal platform team, since AI will touch every part of software. He adds that he is surprised how slowly AI has reached everyday tools. Gmail still cannot create calendar invites from emails, he says, and he would have expected consumer and professional life to have changed more by now.

Masad says it is rational for most companies to wait rather than train their own model, especially as commercial prices keep falling. Replit was different because no commercial model offered the characteristics it needed, chiefly low latency and low cost. He notes that GitHub Copilot reached the market through a deep partnership with OpenAI that gave it custom models, and he says it has since become slower.

Replit wanted AI to be part of the free experience, and when it ran the numbers, being a customer of commercial models never made sense. It prototyped with Salesforce's 3-billion-parameter model, found that small models are capable, and then spent about $100K training its own 3B models. The AI team is only seven or eight people inside a company of 100-plus, and Replit still uses commercial models for other features such as general chat.

His advice is to start from the customer problem, explore solutions and run the numbers. He adds a strategic angle: if you consider yourself an AI company, building a competent AI team has value even if you mostly use commercial models, as he says Perplexity does.

Masad recalls a viral tweet arguing that we do not have true open-source models, because you cannot reproduce them. His analogy is that having only the Linux binary, or the source without the compiler, would not count as open source. Anyone using such models depends on Meta continuing to release new Llama versions, and while he respects Meta and Mistral, he does not want to bet a business on that goodwill.

He therefore thinks companies should treat open models much like commercial ones: prototype and maybe deploy them, but plan for a sustainable path. For now he sees the commercial side as ahead and expects most people to depend on it, with AI labs increasingly resembling enterprise companies with solutions teams and custom models. He would love to see a genuinely open project that accepts contributions and has a real open-source flywheel.

Since data is the source code of a model, he says, not knowing the training data is also a security risk. He recalls Ken Thompson's "Trusting Trust" paper, where a backdoor in a bootstrapped compiler evades source inspection, and suggests an LLM could hide behavior triggered only by a particular phrase. He then muses that the world is getting weirder and will need tools to tell real media from fake, recalling a video that briefly fooled him of Kim Kardashian and Taylor Swift teaching calculus.

Masad says background agentic workflows, such as asking a model to refactor code and run the tests, get expensive quickly with GPT-4, so Replit has not invested heavily there beyond research on more effective agentic models. He is surprised that LLMs have agent capabilities at all, having expected action Transformers or large action models to be the route. For pricing, he advises thinking in terms of value rather than cost-plus, projecting that inference will get cheaper, and noting that some model providers are effectively VC-subsidized.

He bet years ago that usage-based pricing would become more prevalent, since even tiered plans cost money on heavy power users. Replit offers a bundle with overages, and he invested in a company that provides usage-based billing as a service. The hosts add that costs will balloon once models run in CI/CD pipelines and while developers sleep.

On agents, Masad thinks they are the next big step, more than multimodal, which he sees as an incremental improvement. His advice to founders is to walk through walls and build with whatever exists, even if it is an expensive GPT-4 prototype, while established companies with little AI talent can wait. A first milestone would be giving a model a bulleted list of actions without elaborate scaffolding and having it follow them reliably. He notes that function calls work about 90% of the time, but the 10% failures can be catastrophic, and he cites Sweep's claim of around 30% accepted pull requests as a metric to watch.

Masad says the default assumption is that Microsoft wins AI coding, given its install base, enterprise reach and sales team. The optimistic case is a new crop of companies that specialize in parts of the coding workflow, such as test generation. He criticizes Copilot for being locked to the editor and lacking context on other repos, collaborators, git history or Jira tickets, whereas Replit aims to put AI on top of an entire cloud development environment.

On code-generation startups such as Poolside, he calls the bet to build the best code model worth making, while acknowledging the pessimistic view that a GPT-5 leap could undercut the economics. Replit sees itself as an applied AI company that will do research and training when needed but focuses on delivering value. On Code Llama, he has not tried it himself and likes to sit with a model and get the vibes, and he notes the early vibes online were mixed even though benchmarks reached GPT-4 level.

In the quickfire round, Masad calls chatbots overhyped, since some things should not be chat interfaces, and says the underhyped idea is using LLMs as part of the call chain of ordinary backend systems. His biggest surprise was how much latency matters: waiting two to three seconds instead of 300 milliseconds changes the UX entirely. A feature that flopped at first was inline contextual actions, which Replit now promotes and sees growing.

He admires OpenAI for its ambition and Sam Altman for spinning many plates, and he is bullish on Perplexity because of the engineering behind it. Asked how engineering teams will change, he thinks that within five years a company could do today's work with a tenth of the engineers, and within ten years companies could shrink dramatically. Software creators will keep growing in number, he says, but might be called something else, as TikTok stars are called creators.