The Real Bottleneck in AI Research Is Compute, Not Ideas

Jacob Effron of Redpoint Ventures reconvenes Ari Morcos and Rob Toews for a year-end session of Unsupervised Learning, a week before the holidays, with NeurIPS still in their bones. Morcos was counting: this was his eleventh NeurIPS. The first, in 2015, had about 2,000 people. Last year 22,000. This year 30,000, more than four times the entire attendance of the year he joined, fifteen times the room he walked into as a neuroscience PhD trying to break into machine learning. The feeling then, he says, was that he was late. AI had already happened. Students he talked to on the floor this year had the same anxiety, scaled up: the conference is huge, surely we missed it.

His verdict now is the opposite. The field is still extremely early. Thinking he was late in 2015 looks ridiculous in hindsight, and he expects people to say the same thing five years from now. What has changed, uncomfortably, is the work on the posters. There is less exciting research at NeurIPS. The conference has become more about PR than about sharing, and the posters are not only six months stale, which was always true, but incomplete. A lot of the best work is simply not submitted.

Meta and Google now run publication policies with a Goldilocks zone: the paper has to be good enough to get through review, and not so good that it actually matters for the model, or you cannot publish it. Researchers who want the line on their CV strip the large-scale results and submit only the small-scale ones. Want the real curve? Join the company. Effron notes that the event filling up with VCs is highly correlated with a relative decline on the research side. Morcos will not let the labs off: it is a combination, and investors are not blameless either.

The question hanging over the last month is whether models have plateaued and whether scaling laws have run out. Toews’s answer is yes, and he does not see how you disagree. The leap from GPT-1 to GPT-2, GPT-2 to GPT-3, GPT-3 to GPT-4 had a breathtaking slope no matter how you measure it. Since GPT-4 the increments have shrunk, which is what S-curves look like. Everyday tasks — a travel recommendation, pet care — already get as good an answer as there is. Continual learning and sample efficiency, the fundamental gaps versus human intelligence, are not being closed by the current stack. Freeze the models tomorrow, he still thinks there are trillions of dollars of commercial value to unlock, because pushing this into the enterprise takes years. The slowing of model progress is still, in his view, a reality, and it is why other research approaches are blossoming.

Morcos splits the question. He saw a tweet he liked: we are not in an AI bubble, we are in an LLM bubble. The community substitutes language models for AI far too easily. Are AI models plateauing? Almost certainly not, if you leave LLMs out. Video models over the course of this year are a hard case for anyone who wants to say the field stalled. Consumer LLMs, on the usual can-it-answer-the-test-question metrics, do look flatter. The thing that still drives him crazy is sycophancy. He wants a model that will argue with him, and it is still very hard to get one to do that well. Toews will not separate enterprise from consumer on the capability question. Enterprises are not getting a different base model. The remaining work is productization, integration, security reviews. Foundational capability, he says, has plateaued in both settings.

Effron’s enterprise picture is narrower and bleaker. The use cases that clearly work are internal document search and coding, with customer support a weaker third. Beyond that he does not see many huge deployments. When he asks why, the answer he hears most is accuracy: the models are not good enough at the task. He does not think that always requires new research. Applying techniques and know-how already in hand could close a lot of the gap. Getting there is just really hard right now. The live question for 2026, in his framing, is whether good reinforcement learning inside those domains unlocks anything, or whether the capabilities are already there and the job is chaining, prompting, and process.

Morcos has been thinking about RL as a Goldilocks problem, not a panacea. Teams treat pre-training, mid-training, post-training, and RL as separate orgs, customers of one another, when it is all training, and the pattern that keeps working is dragging later-stage tricks earlier. RL pays off when the model knows enough that on-policy guesses have a chance, and not so much that the task is already easy. Too hard and the policy does nothing useful. Too easy and you harvest reward without teaching. Coding has reached that edge. Other domains have not, even though RL is not a one-trick pony. Two years ago everything was RAG. RAG is not dead, and it will sit in many deployments, and it is not the answer to everything. He expects the same sentence about RL. The next research question is how you get a model *to* the zone where RL can work, including for computer use, where some internet-trained models are already ready.

Toews names three research vectors he will watch in 2026. Continual learning: drop the split between training and inference, let weights update on the go the way a human does. Labs currently kludge this with ChatGPT memory and periodic fine-tunes. Recursive self-improvement: an AI that builds a better AI, the old fast-takeoff story, now a real lab focus, with OpenAI talking publicly about a working prototype of an AI researcher next year. And data efficiency. Zoom out and the contrast is absurd: gigawatt-scale data centers that draw as much power as San Francisco, months of training, hundreds of millions of dollars, versus a brain that sees a few examples and learns. Breakthroughs could come from hardware, including whatever Naveen Rao is building, or from algorithms.

Morcos wants to slow the AI-researcher story down. The pitch assumes the bottleneck is that we do not know which ideas will work, and that a synthetic researcher finds them faster. He is not sure that is true. He doubts OpenAI lacks, somewhere in the pile of proposals its people have already made, a very good idea for the next model. The cost of experimentation, time and bandwidth, is the constraint. Writing the code is no longer the bottleneck. GPUs are. A strong AI researcher who generates lots of ideas still has to test them. The only way the loop explodes is if the machine’s hit rate is much higher than a communal org of strong humans. That is a higher bar than “more brainstorming.” The analogy is drug discovery, where current hit rates sit around 0.0001 percent, so lifting them to 1 percent is a ten-thousand-fold gain and AI has somewhere to stand. He is not convinced AI-research ideas have that kind of headroom. What *would* change the game is small-scale experiments that actually predict large-scale ones, and an AI researcher that found *that*.

That argument is his on-ramp to the sharpest claim of the hour: the fact that labs can raise infinite capital is a huge detriment. Look at what China has done with far fewer resources than any individual Western lab. xAI alone, he guesses, probably has more funding and compute than all the Chinese labs combined, and they keep shipping. Constraints breed innovation. Kaplan-style scaling laws already said that every 10x of data buys a diminishing return. That is what a power law is. If you have infinite money, buying more data and throwing more compute at the wall is a viable strategy, and faster than solving sample efficiency, which is the hard problem Datology actually works on. They report order-of-magnitude sample-efficiency gains and much better models on the same compute budget. They have to, because they do not have billions to spend their way around it.

Ilya Sutskever’s line last year at NeurIPS, that pre-training is dead, rested on the internet being a finite resource already exhausted. Morcos thinks Gemini 3 has explicitly proven that wrong, and that Google did a lot of work on the data side. The assumption that we have squeezed all the juice out of existing datasets is, he says, patently false. Hand-juice an orange and you get a pitiful amount for a lot of work. Put it in a juicer and you get three times as much from the same fruit. We are in the hand-juicing era of modern datasets. Synthetic data is now table stakes. Chinese labs, denied the expensive path, have been innovating on file systems, curation, and other unglamorous plumbing. The good news is that the West has not done much of this yet, which means there is headroom.

New labs are spinning out around continual learning and recursive self-improvement. Toews thinks the incumbents are well placed to lead those problems, and that more groups working on them is still good for the ecosystem. The steelman for a neolab is innovator’s dilemma: OpenAI, Anthropic, and Google have a stack, incentives, and an org that pull everyone toward more scale, the easy path Morcos just described. A startup can start from first principles. He still thinks the most likely outcome is that the breakthroughs come from the big labs, or leak. Silicon Valley is porous. A researcher leaves, the secret gets out. The fascinating exception is Safe Superintelligence, Ilya Sutskever’s company. As far as Toews can tell, literally nobody outside knows what they are doing. Sutskever said interesting things on Dwarkesh Patel’s podcast and still did not hint at anything concrete.

Morcos says SSI’s secrecy goes beyond anyone else. You cannot bring a phone into the office; it goes in a school-style Faraday bag. Staff are not allowed to comment on AI, period. They could not have this conversation, because an opinion on the latest Gemini model might leak something. That is official policy. He will not count Sutskever out. In 2018 Ilya tried to convince him that scaling was the way. Morcos said absolutely not, you will not get compositionality from scale. He was wrong, the bitter lesson was bitter, and that is a lot of why he moved to data: inductive biases were not the answer, compute and data were, and he did not know how to make GPUs go brrr. People with that kind of track record, Sutskever and, he adds, Mark Zuckerberg, you do not casually fade. The other side of the secrecy is a fork: either they have the next big thing and have kept it, or they do not have anything and there is nothing to leak. Aggressive secrecy has that downside. You cannot tell which world you are in.

Effron has an unattributed anecdote at caricature-level mystique. Someone interviewed at SSI, did not take the job, learned nothing specific, and still came away with a line from Ilya: there are two words which, if I told them to you, you would understand what SSI is building. I cannot tell you what those two words are. Toews, without pausing, spoils it: continual learning. Morcos turns the neolab question around. Continual learning is a real problem, though consumer memory kludges are pretty good, and he often wants an incognito chat so the memory will leave him alone. The bull case he wants named is the business, not the science. Do they invent the thing and get bought, or is there a path to an independent company? Toews’s version is a three-to-five-year new paradigm: whoever gets there first sits down at an oligopoly table that, in the current round, already prices at a $300 billion minimum. They will not be alone. Ideas spread. Being first still might be a decent shot.

Morcos takes the case apart. OpenAI and Anthropic rose on a brand-new product surface. Many neolabs, as he understands them, are still aiming at the same chatbot. A chip company is a different surface. Lila Sciences or Periodic Labs, going after scientific discovery rather than the big labs’ problem, is a bull case he can see. A better consumer chatbot is not. Enterprises are two or three years behind the frontier; they will not pick a startup over OpenAI because it is three months ahead. o1 shipped in September and strong open reasoning models existed within three months. He used to believe the great-man theory of science. Wikipedia’s list of multiple independent discoveries cured him. Calculus is Newton and Leibniz, and that is the rule, not the exception: every major innovation was found by at least two people within a year or two, until communication got fast enough that you could not miss a rival’s paper. Reasoning models spread so quickly, in this telling, because every lab was already trying the idea. OpenAI got it working a little sooner. Existence proofs help. Leaks are not required. Toews still thinks you can build a brand around being first and take that to market.

OpenAI called a code red in recent weeks. None of them think the company is doomed, and all of them think the coverage has been overplayed. Toews’s version is that a uniquely charmed firm, able to do no wrong since ChatGPT, is coming back down to earth. People are noticing it may not have irreplicable sauce. Google, on talent depth and compute, looks better positioned on many dimensions. ChatGPT is still the defining consumer product of the generation. The aura of infallibility is what cracked. The scarier point is cash. Google has hundreds of billions on the balance sheet and throws off tens of billions a year. OpenAI is burning an insane amount of money and exists at the pleasure of capital markets. A chart of cumulative capital burned before cash-flow positive put Uber at about $40 billion, Tesla at about $10 billion, and OpenAI, by its own investor projections, at something like $150 billion, with profitability around 2029. Sam Altman is a genius at fundraising. If investor sentiment turns, OpenAI faces a spot Google never will.

Morcos reads the code red as the end of the beginning, closer to Zuckerberg’s “we are going to war” memo when Google+ launched than to a death rattle. A year and a half ago OpenAI’s model was obviously better than anything closed or open. Now Google and Anthropic are close, if not better, on many tasks, and Chinese open models are as strong as ChatGPT was less than six months ago. Google is on the ninth or tenth generation of TPUs, a project that started in 2016: not that flexible, extremely good at what they are good at. In the long run he expects the most-used models to be the ones that ship as defaults on phones. ChatGPT’s brand has already escaped ordinary product risk. Apple Maps still took a lot of share from Google Maps over time. He expects the same pattern here, not overnight.

Disney’s exclusive partnership with OpenAI, putting IP into video models, looks to both of them like a harbinger of how image and video models differentiate: licensed worlds, the same script Suno has started running with labels. Toews thinks it is smart on both sides. Disney invests at a favorable valuation and, instead of leaving a void for unlicensed generation, offers a permitted path, having apparently learned from the long, piracy-friendly pause in the streaming transition. Exclusivity is presumably time-limited. In two or three years they can switch. Thrive Holdings is a different kind of deal. OpenAI is a model company, not an implementation shop; it has already used McKinsey, Bain, BCG, and a Palantir-like startup called Distill to extend reach. Morcos thinks the real payload is data. Public evals are close to useless. When a new model drops, the evals that matter are what Cognition thinks, what Cursor thinks, what the people in the weeds think. A holding-company partnership is another sensor on whether the models are trending in the right direction, including in unsexy non-tech industries.

Effron asks for an update on Meta’s superintelligence effort. Morcos is more bearish than on the last episode. MSL has already seen departures after extremely short tenures, plus cultural divide. The big layoff looked to him like a purge of everyone who had touched Llama 4, including exceptional researchers who should not have been in that pile. Paying a fortune to hire AI talent and then letting good AI talent go for organizational reasons is, in his word, crazy. FAIR, where he spent some of his favorite years, looks to him like it is on its last legs. Everyone he talked to from FAIR at NeurIPS was unhappy and looking for an exit. New models are probably going closed source. Llama’s tailwind was the open-source story, which, along with Zuckerberg’s makeover, rehabilitated his public image. A Meta model as good as ChatGPT, now closed, may not get the same goodwill. Mercenaries, drama, and org chart fights do not make a strong model easier to build.

The chip story is the most contested. Toews rewinds to the Biden years: the United States noticed it had a choke point in Nvidia-designed chips, used it unapologetically, and it worked. Total cutoff, though, forces China to build a domestic industry, constraints breeding innovation again. China’s chips are not at Nvidia’s level, and they are improving because they have to. The Trump administration, with David Sacks in the mix, opened the spigot a little: sell H200s, decent but not Blackwell, good enough to compete with Huawei and slow the homegrown stack. China may simply refuse the H200s for the same industrial-policy reason. Nvidia keeps supremacy for the next few years. The CCP thinks in longer timeframes, and over a long enough horizon an internal industry is to China’s advantage.

Morcos agrees and then goes further. Forcing China onto handicapped chips will be remembered as one of the stupidest things the United States has done, a short-term foot-gun that still treats China as a 1990s copycat factory. That picture, he says, is so clearly not the China of the 2020s. They are not fast-following in AI. They are innovating. A new DeepSeek paper is must-read at every lab in Silicon Valley. They are innovating under the model layer, on infrastructure, the unsexy plumbing that actually raises iteration speed. He guesses the bans accelerated China’s chip development by at least several years, and five to ten would not shock him. If the West had sold chips from the start, China might never have bothered. Toews does not buy the full counterfactual: the choke point was too obvious, and urgency would have been there anyway. The variable he does buy is whether Beijing forces model labs off Nvidia. Once those labs have to train on domestic silicon, the feedback loop tightens and the chips get better fast. That mandate, he thinks, is probably happening several years earlier than it otherwise would have.

Amazon’s Nova Forge announcement excited Morcos because it matches a Datology bet he has held for years: most enterprises will end up with a bevy of specialized models, not by training from scratch, but by customizing open weights on their own data. In 2024 that was non-consensus. One model to rule them all, plus light fine-tuning, was the fashion. Small fine-tunes disappointed, and people had to go to larger training runs. Amazon and Azure see what enterprises actually need. The detail he likes is the checkpoint. You do not want to continue-train from a fully annealed, overbaked base. You want an intermediate pre-training checkpoint. Most models, including all the Chinese releases he mentions, only ship the final one. Nova Forge works with intermediate Nova checkpoints. He reads that as Amazon understanding the job, and as a tell that 2026 is when enterprise AI beyond search, code, and support actually starts to work.

Year-end surprises first. Toews’s is a personnel event: Daniel Gross leaving SSI for Meta, and to a lesser degree Nat Friedman. SSI’s whole culture was loyalty, no leakage, everyone in the family. Gross was close to Ilya. A Zuckerberg paycheck, Toews thinks, is not explanation enough for a man who already has money. Effron’s needle: Gross probably knows the two words, which makes the move harder to parse, not easier. Morcos’s surprises are Meta’s full about-face on Llama — Llama 3 had been a de facto standard, Llama 4 underperformed expectations more than it was a disaster, then came the purge and Alexandr Wang — and, larger, the rise of Chinese models. This time last year DeepSeek V3 was not quite out. R1 arrived in January. Then Kimi, Meituan, MiniMax, all in the open. Going into 2026, the paragons of open source are basically exclusively Chinese companies. He would never have predicted that. DeepSeek V3 was less than a year ago. It feels like five years of progress.

What they changed their minds on: Toews no longer assumes the big labs stay private. Anthropic has hired Wilson Sonsini and is thinking about bankers. Both firms took crossover capital that will want a liquidity event, and the businesses are starting to look like real companies, unprecedented burn included. He now expects IPOs in 2026 or 2027. Morcos agrees, and he has also gotten more bullish on RL than his own podcast persona suggests. RL has been the boy who cried wolf for fifteen years, including DeepMind’s 2014-2015 bet, which was early. Last year the field was still in the RLHF era; a lot of that died quickly and turned into SFT. RL has since generalized and produced practical gains beyond what he would have bet. It is still not a panacea. Relative to history, it now works in real settings, which you could not say a year ago. Effron notes that on the AI podcast circuit, a relatively bullish take still lands you in the bottom quartile. Andrej Karpathy saying agents will automate everything, but it will take a decade, got treated as disbelief.

Then the holiday treats. Toews previews a Forbes column: by the end of 2026, Sam Altman will no longer be CEO of OpenAI. Gemini 3 changed the narrative; OpenAI is on its back foot for the first time. Altman is everywhere else, a BCI company called Merge Labs that OpenAI is putting money into, talk of buying a launch company, own chips, own data centers, consumer hardware with Jony Ive. That looks ambitious when the firm can do no wrong, and unfocused if ChatGPT growth slows. Fidji Simo is already CEO of applications and running a lot of the day-to-day. An IPO path needs a CFO, quarterly reporting, trains on time, a Travis-to-Dara shift. If it happens, the public will hear it as Sam’s decision. Morcos disagrees: AI is the top of the first inning, and swapping in a short-term optimizer is the worst move you could make. His own bet is hotter in a different direction. Greater than 50 percent chance that at least once in 2026, the clearly best model in the world, by consensus of people in the know, is a Chinese open-source model. Greater than 20 percent that at year-end the unequivocally best model is not from a closed lab. Effron’s is on the capital side: 2023 through 2025, application-layer value crushed infrastructure, data labeling and inference aside. In 2026, with the substrate finally stable, more value shows up in infra, including categories like search tools that plug into agents. Morcos, as an infra founder, could not agree more, and still wants more spice.