ChatGPT Runs on His Framework. He Still Doesn't Want You Training a Model

Sarah Guo and Elad Gil welcome Guillermo Rauch, founder and CEO of Vercel, by pointing at a simple fact of the public web: a lot of it already runs on his company, and more of it is about to run on Vercel plus AI. Rauch describes Vercel as a web infrastructure company. It sells the frameworks, tools, infrastructure, and workflows for deploying ambitious, dynamic sites. The technology behind ChatGPT’s interface, he says, is powered by Next.js, Vercel’s open-source framework. Under Armour and Nintendo sit on the same infrastructure, which serves their traffic and lets them iterate on the site.

He started the company at the very end of 2015 and settled on the idea, with the first prototypes, at the beginning of 2016. There is a duality in him, he says. On one side he is a missionary of the open web: he wants the web to win, open platforms to win, developers to win. On the other he loves Apple, and companies that bet heavily on design, integration, and ease. The founding bet was to do for the cloud what the iPhone or the MacBook did for personal computing. He had just sold a previous company to Automattic, the firm behind WordPress, and he wanted deploying an idea to the global web to be so easy a developer did not have to think about it.

Frontend had been his strength for most of his career, and engineering culture, he says, had treated it with something close to disdain — the last thing you worry about. Vercel turned that upside down. Frontend is the most important thing a company has, because that is where it meets the customer, where a faster site drives conversion, sign-ups, and sales. He wanted a company that started at that last mile of end-user experience and worked backwards into the integrations and backends a full-stack app needs. Vercel, to him, became a portal into the web and into a new way of building software.

Rauch has been an AI fan for years. As an angel he was an early investor in Scale AI. To him, AI is another large step in automating the parts of a creative endeavor nobody wants to deal with, which is true to Vercel’s spirit: putting any new backend into a dynamic site should be absurdly easy. The sharper insight is that foundation models feel like cloud 2.0. Tremendous SaaS businesses were built on AWS; Snowflake is the example of a great cloud-native company that did not have to reinvent the infrastructure. His new version of that sentence is that a lot of great AI-native companies will be built on this new infrastructure. The models are the new backends that will power the most exciting frontend applications.

That is why Vercel shipped an AI SDK, which he says is now powering a ton of startups, including many in the company’s AI accelerator. The pitch is the easiest way to create an AI app without reinventing the backend wheel: connect to OpenAI, Hugging Face, Replicate. He is not for “random acts of AI,” checking a box in a report. He wants useful products, and he believes a lot of the value accrues in that last mile of integration. On the roadmap the SDK is text-oriented today, with voice, audio, and image generation planned so frontend engineers have a wider toolkit.

The hard infrastructure underneath is about latency that traditional backends were never designed for. Most e-commerce systems optimize to respond within 100 milliseconds. An AI model can take 15 to 20 seconds to fully bake a thought. Users will not stare at a blank page for that long. Vercel’s Edge Functions run compute as close to the visitor as possible and are built for long sessions of dynamic streaming, so tokens can arrive one after another from the edge of the network. On the SDK demo page they show the gap: a blocking traditional backend versus streaming. In the template marketplace, apps like RoomGPT — redesign a bedroom with an image model hosted on Replicate — come with login, signup, and a turnkey subscription, deployable in seconds. The through-line, he says, is putting AI into as many developers’ hands as possible.

Looking ahead, Rauch thinks the integration side and AI-native products are still underexplored. For founders listening, he does not believe you have to train or fine-tune a new model to make something legitimately useful. He has been watching Jenni.ai: a specialized product that helps researchers write papers, which went from $1 million to $1.5 million in ARR over two months. What they are doing, in his telling, is starting from a problem that already existed and asking what the solution looks like if AI is now an input in the design space. That is a radically different stance from adding AI to a word processor. A lot of the productivity tools he uses every day, he says, could be rethought from the ground up. Vercel’s advice is to start with the frontend and the AI SDK, which saves a pile of integration time.

The company tries to be customer zero of its own products. The entirety of vercel.com is built on the Vercel platform, which pushes them to make Next.js better, the infrastructure better, and builds faster as headcount has gone up. They want the same loop on AI. If you could automate the daily work of frontend engineers — forms, UIs, layouts — a lot of it is statistical. A good interface has to be familiar; you cannot pursue a wholly original journey every time you sit down to make a new frontend. They are dogfooding the AI SDK against that next frontier of generative automation, applied to the domain they know: UI and frontend engineering.

Guo notes that Vercel is one of the most loved developer-centric companies going. What, she asks, is still missing in AI developer tooling? Rauch points at instrumentation. The winning observability products of cloud 1.0 were born from a simple move: we understand the frameworks and primitives you already use, and we integrate extremely well with them. The first time he used Datadog, onboarding would not let him leave the page until he had sent a data point, and instead of a foreign API it enumerated integrations he already knew — Kubernetes, Next.js. He recently went through Zapier’s signup from scratch. Zapier runs zapier.com on Vercel. After signup it asked what software he used and which integrations he needed, and he could not even click the Zapier logo because he was already that deep in the funnel. He thought it was beautiful. That pattern, he says, has not fully landed for AI. The topology is different, the frameworks are different — the AI SDK, LangChain.js, a pile of new ones — and the things you monitor are different too.

Gil agrees the monitoring, testing, observability, and feedback-collection space is nascent and important. In cloud 1.0, observability was a nice-to-have you obviously needed for a production app. In the AI realm Rauch calls it mandatory. Your v0.1 already needs that critical feedback loop. Early-stage engineers moving fast used to be lenient about how much they observed their endpoints. Model output is unstable; without monitoring you do not know how the product behaves for users.

His other hot take is that a lot of the early, opinionated AI frameworks will have to evolve a great deal. He expects a second generation to come out of actually building and deploying AI at production scale. The DX tools so far are rooted in getting the job done, not in knowing whether this is the best way, because the apps have not been in production that long. He is not talking about training tools. PyTorch is well baked. He means the last mile: everything to do with agents, indexing, retrieval, and the novel integrations of AI applications.

Gil looks ahead to a web where a subset of interactions become agent-based: an agent for you, for a company, for a government, acting through APIs. Does that even matter for Vercel? Rauch says it already matters tremendously, which is why they are investing in security products. When GPT-3 landed and everyone rushed to integrate it, OpenAI became, by Vercel’s aggregated anonymous telemetry, the most popular backend that server functions talked to. Then the abuse began. People stood up proxies at scale to extract free intelligence. He calls it scraper 2.0: instead of writing a page scraper, you run a bot that tries to get free GPT-4. It is still a huge problem. Products that integrated AI in a generic way have opened their token pool to the entire internet, authentication or not — including countries and companies where the AI is already banned.

Vercel is giving developers tools for that: rate limiting, bot detection, and caching of OpenAI responses that can be cached, which also cuts the cost of deploying the AI. The second problem is distinguishing a good bot from a bad one. Website owners at scale need a sense of the right ratio. A lot of AI companies, afraid of abuse, are already very strict about blocking any bot activity. The arms race, he says, is AI versus counter-AI: deploy more and better models to keep the bad bots at bay while still letting through the authenticated agents that will crawl the web on a user’s behalf. Every customer that has shipped a working AI product at any scale has already faced this. Sometimes it is cat and mouse; sometimes Vercel is just teaching the customer how to put the protections in.

SEO is loosening at the same time. The old conception was optimize keywords for Google and make the site fast so the crawl boosts you. There is already an intermediary that is not Google — GPT-4, with its dated corpus — and now Perplexity doing real-time retrieval. Customers keep asking whether Google is still king and whether the old SEO rules still apply. His own pain is concrete: ask ChatGPT a Next.js question and it gives you the 2020 answer. He wants 2023. “Help yourself to our docs. I can give you whatever format you want.” For other businesses, content negotiation will be a fight, closer to Getty Images pulling data from image models. He also expects today’s APIs — a stream of words that answer the prompt — to become a duplex stream of content and citations. Products, and maybe the law, will need to log where an answer came from. Users may want to hover a sentence and see the source; companies will want to know which sources users keep coming back to.

Gil notes how expensive a fresh index of a trillion URLs is, and how agents will need a web they can actually use. Rauch’s idea is to delegate full coverage to the large models’ pre-training, then complement it with a small, up-to-date index of the sources that matter for your domain. He uses Phind, a Vercel customer that focuses on high-quality developer results; for tactical questions about a vendor it has been excellent. The same pattern, he thinks, will show up for knowledge workers in other fields, which makes the web a lot smaller. Google’s open CrUX dataset — Chrome User Experience Report — is anonymized telemetry of the highest-traffic sites. ChatGPT and Character.AI are already in the top 1,000. The distribution is a brutal power law: the top 1,000 sites amount to about 50 percent of page views, even more slanted on mobile than on desktop. A crawler that only targeted the top 10,000 would still cover what most people actually use.

Asked where web architecture itself is going, Rauch is done with the JAMstack static turn. Content already changed fast; rebuilding the entire site on every CMS edit produced a web where, in 2023, getting a change live at scale might take an hour, through layers of cache and a huge build. A lot of the work behind Next.js has been dragging the industry from static to dynamic. Now every headless CMS vendor is adding AI, and so is Notion. If the rate of content production keeps rising, the need for dynamic infrastructure only gets more urgent. Personalization comes with it: more of a web that is just for you, delivered quickly.

The frontend meme is that engineers change tools every week based on Hacker News. The CrUX crawl of what large sites actually use says the opposite. React, Svelte, and Vue look here to stay; React dominates at the top of the web. He does not expect much framework churn. The innovation moves to the AI tools that generate that code. Midjourney works because almost every prompt yields something statistically pleasant. Building for the web, he thinks, will go that way. The two current extremes are hand-placing every pixel from a blank canvas, or using templates so every startup looks the same — the purple-ish page with a gradient headline and three boxes. AI can give both the easy start and the stochastic novelty models are good at, then let you refine from taste. He recently tweeted the Rick Rubin line: artists hire him because of his taste and his confidence in what he likes and does not like. He sees the product-engineer role evolving into that loop — I like this, I don’t, re-prompt, this looks too much like the average website — with a dive into code only when needed. Existing tools are enough. No breakthrough required.

Gil has heard claims that 40 percent of the code in Copilot-associated repos is generated by GitHub Copilot. Will human languages give way to more efficient machine-driven ones? Rauch treats it as a productivity race: you have to meet the world where it is. Copilot’s success is that it met him in VS Code, and even shipped a Neovim plugin. He believes the generation numbers because developers hate the liability of a third-party package. Every dependency contaminates the supply chain with a bag of surprises. The last 10 or 15 years were the rise of the package manager — Python, Ruby, JavaScript, Rust’s Cargo — but underneath, we have been copying strings from the nearest CDN onto a laptop. AI makes copy-and-paste so ergonomic that you have to ask whether you still need the package. In UI this is already the case. Win32 and Java Swing libraries ate huge investment and then nobody used them, because everything looked the same. The most popular way to build React UI now, he says, is shadcn/ui, whose author told people to copy from the browser into the editor. That was the breakthrough. Copy and paste is always better than a bad abstraction. The worst codebases are the over-abstracted ones.

That is also an argument for owning the code. If you vendor it into a monorepo you can optimize and secure it, and you are not stuck overriding a vulnerable third-party package. Vercel’s Turbo exists because the companies that have written the most successful software work in massive monorepos, not in 99 repositories you hop between for each feature. Those companies do not lean on global package managers: too much liability, and it is easy to paste the code in, assume ownership, and audit it. An AI that scans the repo then has full visibility of every dependency on the critical path. Supply-chain tools like Snyk and Socket, and a security Copilot that audits autocomplete in real time, get more useful in that world. As for machine-derived languages, GPT-4 is extraordinarily good at JavaScript and Python because the internet is full of them, and not very good at CUDA. That is a four- or five-year question. What is immediate is token efficiency: redundant HTML closing tags are a waste, and Vercel has already built simpler layers of logic that map back to the format you actually need. He closes by pointing people at Vercel’s AI section and nextjs.org. A first AI app, he says, is minutes away.