Karpathy Called Today's Models Slop. His Decade Timeline Still Sounds Bullish

Jacob Effron is back on Unsupervised Learning with two recurring guests: Ari Morcos, head of research at Datology AI, and Rob Toews of Radical Ventures. The prompt for the conversation is a collision of headlines that make the bubble debate suddenly specific. Andrej Karpathy recently said the models are not there, the industry is jumping too far and pretending the results are amazing, and the output is slop. He also said it will take at least a decade before AI can automate entire jobs the way an employee or intern would.

In the same window, OpenAI has been hiring bankers and paying them to train models on financial modeling and spreadsheets. One camp talks as if AGI is around the corner. The other is paying domain experts to teach a model the entry-level work of an investment-banking analyst. Effron wants a researcher and an investor to walk the month's news without the usual lab talking points, from Karpathy and capex to Atlas, Sora, and the application stack.

Morcos's first reaction is that Karpathy is still quite hyped on AI. Treating "AGI in ten years and all work automated in ten years" as evidence of a bubble is, in his view, a crazy statement. Had someone told him five years ago, with high confidence, that all work would be automated in fifteen, that would have been the biggest technological advance ever. The fact that this now counts as a conservative take is mostly a comment on how overhyped the discourse has become.

He does not deny the overhyping. He talks to people who think they have two years to make money before nobody makes money, because everything will be fully automated. That is obviously false, he says, if only because enterprises take a geological age to absorb anything. Big companies are slow, and real-world impact will lag the demos for a long time. Even if models stopped improving today, the economy would still see a massive transformation. This is a splash of water on the wildest claims, not a comparison to the dot-com bust.

Toews loved the interview for the same reason. Karpathy is not bearish and is not calling AI a bubble. He is naming the places the systems still fall down, and he is saying we are not in a six-month fast-takeoff. Hearing someone of his stature say that out loud felt refreshing. On the labs' "year of agents" line, Morcos says it matches what many researchers say privately. Social media rewards extreme takes: "we're one year away" travels further than "there is still a lot of work, especially in specialized domains."

Effron's follow-up is the short-term money question. A long, incredible path can still miss the revenues needed to justify the latest wave of data-center capex if enterprise agents do not show up on schedule. Toews says it has felt like a bubble for two or three years, and many of the signs are real: rounds closing at 2020–2021 ZIRP speed, and valuations to match. He would almost say a bubble is inevitable whenever a paradigm-shifting technology arrives, because people dream the long-term dream and the financial cycle decouples from the technology cycle.

Where it gets more serious, he says, is leverage. Data-center buildouts are starting to take on more debt and larger capital commitments. A frothy round burns investor money. Levered infrastructure can burn the system. Morcos adds a researcher's caution: constraints breed innovation, and he is not sure infinite compute is the right strategy. Naive scale-ups such as GPT-4.5 and Meta's Llama Behemoth did not work that well; mixture-of-experts architectures did, because they are more efficient. DeepSeek reached the frontier on GPUs that were not Nvidia's best. He is unconvinced the industry needs to spend half a trillion dollars on data centers to make AI a lot better.

Almost all current revenue, Effron notes, is consumer chatbots or coding. Morcos points to a sharper hole on the consumer side: analyses that reverse-engineer OpenAI's revenue find only a small fraction of ChatGPT users paying. The "normies" in his own family use it constantly and do not pay. Asked why not, they say, do I need to? They do not know what a thinking model is. Toews jokes that they also do not understand the capex sitting in those buildings. Whether subscriptions scale past a subset of the population is an open question, which is one reason OpenAI is pushing an apps SDK and checkout inside ChatGPT: if the answer engine will not collect, commerce might.

Toews compares OpenAI's Atlas launch to a back-to-the-future moment. Browsers sat still for twenty or twenty-five years; suddenly they are an innovation surface again, with Perplexity's Comet and other startups in the mix. He tried Atlas and expects to leave Chrome for an AI-native browser. Morcos is less sure people will switch. Chrome is sticky, and the winners may be the companies with distribution, mainly Google and Apple. The other variable is whether partners play nice. Uber's logistics network should survive even if ChatGPT picks the ride, but Uber still loves the consumer front end. Shopify faces a blunter threat if shopping happens inside a chat window.

That platform fear, Morcos suspects, is why Mark Zuckerberg is spending so heavily on AI talent. Missing a platform in mobile burned Meta, most clearly when Apple's ATT change made ad attribution harder and cost the company many tens of billions of dollars. If you do not own the platform, it is an existential threat. Oculus was the same instinct for AR and VR; glasses are the continuation. There is a second threat: a lab can cut you off. Anthropic's cutoff of Windsurf is the example that would terrify anyone whose product sits entirely on one model. Toews calls it a strong frenemy dynamic: you do not want to skip working with OpenAI, and you risk being marginalized if you do.

Video is the other zeitgeist hit. Toews says Sora 2 and Google's Veo 3 have recently crossed the point where generated video is near or at parity with realistic appearance; the old Will Smith spaghetti failure mode saturated in a blink. For eight or nine years, deepfakes were a late-2010s panic that the tech could not yet deliver. Now it can, and society has not yet seen the full trickle-down. OpenAI is also making a feature film aimed at Cannes next year in about nine months for under $30 million, against the usual multi-year, much larger budget. Morcos had been more worried about deepfakes than almost any other near-term AI harm and is pleasantly surprised it has not been a major problem yet, including among state actors. He is not sure that lasts. The gray zone is already here: AI video of a politician saying something they did say, just without a camera in the room.

Slop became the word of the month after Meta's consumer launch. Sam Altman said one man's slop might be another man's treasure. Morcos reaches for a hammer: AI is the best hammer ever built, useful for houses and for skulls, with most uses neither good nor bad. Toews notes that social feeds were already full of human-generated slop before generative models. When the cost and time of making text, images, video, and code go to zero, the scarce skill becomes curation. The upside he cares about most is the one Karpathy is spending time on: an infinitely patient, infinitely intelligent tutor in your pocket, especially for people in lower-income countries, if edutainment can be made as sticky as the feed.

Effron puts the banker hiring next to the AGI rhetoric and asks Morcos, the data person, whether the exercise makes models better overall or just better at finance. Morcos is unsurprised. Data acquisition has been moving from generic to specialized for a couple of years, and vendors who can recruit targeted talent are winning. Finance is valuable, digital, and almost absent from the public internet, which is a biased sample of the world's data. His line is that the vast majority of data is private. Enterprises will not share it with frontier labs, fearing the lab will train on it and serve a competitor. So of course OpenAI is paying people to produce the missing tasks. Math and code have already shown that specialized data can lift general reasoning; finance might too.

Toews's medium-term worry is the banker in their late forties or early fifties who went to a great school, worked hard, and did everything they were told. Not all of them lose work, and it will not happen in a year, but a narrower set of people will do the same volume with AI, and the new jobs will need a different skill set. A cohort that feels it did everything right and got screwed anyway is, for him, one of the scarier social outcomes. Effron pushes the other way: senior bankers do not live in cell C137, and Goldman's CEO has said models can already do 95% of an S-1, with humans adding the last 5%. The fragile rung may be the 22-year-old who needed that Excel apprenticeship. Morcos is less worried about juniors, who learn new tools; he is more pessimistic about retraining. After the financial crisis, programs that tried to move auto workers into new skills did not work very well. Willingness to learn new tasks, on average, falls with age.

Even then, Effron doubts banks will call a foundation model directly. A layer of application companies such as Rogo will sit in between, which makes it harder for each vertical to move the needle on a $500 billion valuation or on the capex stack. Morcos has little doubt the use cases work in the fullness of time; the next 12 to 18 months are much less certain, and share prices can wobble even if the long run is fine. Toews adds the physical economy. Robotics trails language models but is improving fast, and today's data centers will train the next robotics models. If those hit in the next few years, he says, we are automatically underbuilding. Not everything has to go right to justify the spend: consumer monetization or high-value enterprise work would each be enough.

The same hedging logic shows up in the labs' deals. Rumors of Google deepening ties with Anthropic look to Toews like a TPU and Google Cloud story, not Google losing faith in Gemini. Anthropic wants compute that is not only Amazon; Google already owns about 10% of Anthropic. Morcos sees complementary endgames: Gemini as on-device chief of staff inside Gmail and Calendar, Anthropic aimed at developers and coding. On OpenAI's flurry of AMD, Nvidia, and Broadcom announcements, Toews is impressed by Sam Altman's dealmaking. The structure makes OpenAI too interconnected to fail. The Broadcom chip partnership points toward custom silicon, maybe even co-designed with each model generation, so OpenAI is less dependent on Nvidia. Morcos's caveat is blunt: every deal is a bet that OpenAI can pay when the bills come due, which is in large part a bet on Altman's ability to keep raising capital. Nobody wants to be single-threaded. Cards get flipped every three to six months.

On the application landscape, vibe coding is the breakout. Lovable, Replit, and Bolt grew fast; Figma entered with Make; Cursor is adding pieces. Some individual contributors at Datology now feel like managers of a swarm of agents. Morcos still sees a lot of human oversight in the loop. Karpathy said the same: he likes models for autocomplete more than for chatting a feature into existence. Morcos expects serious security holes, especially at mid-stage companies, even as longer-horizon coding models get more reliable. Effron splits the use case. As a mockup that is 90% as good as a prototype, he believes it. As a path straight to production, he is skeptical, because most of software is maintenance. If you had no hand in creating the system, who fixes it two years later? Code review, already more common than code writing, becomes the scarce skill: people who can tell when a model goes off the rails.

If Toews had to pick one winner in coding, he would pick Anthropic. Cursor has an amazing product, team, and growth curve, and it is trying to train its own models, but he is not optimistic those will match Anthropic, and the company has a scary reliance on Claude. Morcos is even colder on loyalty. Whoever has the best coding model that day wins that day. His team was all on Claude Code two months ago; most people now use Codex, and they will switch back if Claude jumps ahead. Codebase memory sounds like a lock-in until models get better at inspecting repos, at which point memory can be lifted out as context and the whole pipeline can change models. Engineers, unlike doctors stuck on a transcription tool, always want the new thing. Anthropic and OpenAI also get to run their own models at cost, the way Apple Music does not pay the App Store's 30% fee that Spotify does. Labs can undercut IDEs on price. Open-source options such as Qwen already look pretty good.

Redpoint's forthcoming apps report finds that nearly half of the best AI applications are in San Francisco. Morcos says you can build elsewhere: Toronto has the Vector Institute and the University of Toronto, Zurich has ETH, and you face less bidding against OpenAI, Anthropic, Meta, and Google. Toews's tongue-in-cheek answer is yes, you can build outside SF, but why would you? Capital, talent, and information flow hit the Bay first. If you will not headquarters there, at least keep an office. Effron's illustration is the Cursor team Ubering to xAI headquarters when a new model was failing in the product and fixing it in about thirty minutes. Morcos calls the zeitgeist a double-edged sword: it is hard to go heads down, and it is easy to misread the world. People in San Francisco chase new models. Most of the market is people who are not clued into the AI ecosystem at all.

On data moats, Morcos is skeptical for startups and bullish for large enterprises that have collected problem-specific data for decades. Startups rarely have that. Specialized tasks are where extra examples still matter, and many mission-critical enterprise deployments are not at 95% accuracy waiting for five nines; they are closer to 50%. General-purpose models hallucinate more on specialized work, because hallucinations are often a byproduct of generality, and they cost more to run than a narrow model. LLM-as-judge sounds like a way to get verifiers for free, but Karpathy's warning holds: judges reward the wrong things, such as a long string of numbers, and fixing one failure mode is whack-a-mole. As a neuroscientist, Morcos also pushes back on brain analogies. The brain runs on about 20 watts at a millisecond timescale; going faster costs exponentially more energy. Pre-training is a better analogue for evolution than RL is for childhood. RL only refines a good base model, and Grok 4's headline that half of compute went to RL came with terrible marginal gains versus pre-training dollars.

Asked what surprised him most over two years of AI investing, Toews names the thin-wrapper meme. Two years ago the industry was obsessed with parameter counts and treated anything on top of a model as a shell that would get crushed. In retrospect it should have been obvious that a huge amount of value accrues where products hit real lives. He attributes a lot of the fear to one company, Jasper: a fast rise and a ChatGPT-shaped fall that scared investors off the application layer. He is not sure it has happened to anyone since, and the pendulum swung too far. The other surprise is physical infrastructure. Two years ago few investors were thinking about energy, land, and chips; take the 2020 scaling-law paper to its extreme, and pouring physical resources into bigger models was always implied.

Morcos did expect open models to converge with closed ones on most tasks. He did not expect the success of Chinese models, or the amount of real innovation. When DeepSeek drops a paper, the whole community has to read it. Public perception still says China steals American IP via spies inside OpenAI and Anthropic. That, he says, is patently false as an explanation of the results. Forcing China to buy H20s will look stupid in hindsight, because it catalyzed a domestic AI stack and an earlier move off Nvidia. Constraints breed innovation, and the most exciting efficiency work is coming not from one Chinese lab but from something like ten. US academia is not in the same Goldilocks compute zone: constraints there are too severe, China is funding university compute, and the United States is pulling resources. He likens it to 1960 and the space race, and it scares him.

Effron's surprise is the incumbents that should have won and did not. GitHub Copilot was positioned to own coding, Salesforce to own support, and neither ate the category. Building AI products is a different motion than classical product management. Traditional vendors preview a reliable roadmap and roll it out to the whole company. His AI portfolio companies throw thirty things at the wall, twenty-five fail, three are interesting, two hit, and customers like that experimentation. Morcos agrees the wrapper story is not finished. He expects a bifurcation: vertical wrappers survive, generic ones get hurt, because writing was what ChatGPT was good at on day one. The Goldilocks zone is a market big enough to be worth building and not so big that OpenAI walks in. Effron notes the joke of the moment: a $50 to $100 billion opportunity is now too small for a company that wants to be worth $500 billion.

Apple is the last hyperscaler without a defined AI strategy. Toews doubts it will break a habit of avoiding splashy acquisitions, but conceptually he can see two buys: a DeepMind-style brain trust (Reflection or Mistral are the caliber, and may already be priced out) or a killer iPhone app with a real user base, the role once rumored for Perplexity. Morcos thinks Apple's actual opening is on-device privacy. If each new iPhone runs a stronger local model, hardware upgrades matter again after a decade of people keeping phones for five years. Apple does not need the best model, only one that is a massive upgrade from Siri, which is still the default. Effron worries that competing to be the homepage of daily life means a six-month gap in computer use or life-wide reasoning could move users. Morcos is not convinced. Most people will meet the feature when it ships in macOS, the way they still use Apple Maps. Toews, meanwhile, keeps circling brain-computer interfaces: invasive versus non-invasive, and a bet that the topic goes mainstream in 18 to 36 months. Morcos wants a scaling path that does not require a stack of new nuclear plants. Effron's quieter puzzle is that a year ago the working AI app categories were coding, support, healthcare, and legal, and they are still the same four.