Notion’s Ivan Zhao: RAG Will Make Organizing Optional

Sarah Guo and Elad Gil open with the question Zhao calls the hardest: what is Notion? The product can mean personal notes, trip and wedding planning, or, for companies, documents, tasks, issues, calendars, and a knowledge base in one place. That breadth is the goal. The market, he says, is so fragmented that people should have one tool for most of their work. The method is not to cram every use case into a single surface. It is to find the underlying software building blocks—the Legos—and let users tinker toward a workflow that fits a life or a company.

None of this is new. People tried a building-block approach in the 1970s and 1980s. Notion’s modern spin is cloud and AI, aimed at what Zhao calls “the prison of application-based software.” Why don’t more people try unified tools and unified data? He counts several angles that keep coming around. No-code comes from power users who want everyone to modify the software they use every day. Procurement comes from the budget: pay one vendor instead of five. Language models add another: knowledge in embedding space wants to live in one place.

Notion’s own angle is literacy. People spend years in school learning to read and write, then treat the same MacBook as a typewriter and a YouTube player. There is a wall between those who can make software and those who use it, which is why San Francisco rent is so expensive: “we’re the modern day Detroit or Manchester,” the factory of the software world. In college Zhao read Douglas Engelbart’s *Augmenting Human Intellect*. Software in the 1960s through 1980s was more malleable; in Smalltalk you could change how the operating system worked on the fly. He wanted a breed of software people could modify, bringing back some of that pioneer ethos.

The hard lesson, he says, is that most people do not want to create software. Nobody wakes up eager to build a perfect project-management tool. The boss asks for something, and they have to get the work done. So Notion pivoted: package the bricks as ready-to-use templates and use cases people can adopt quickly. Text editing is one brick, a relational table another, plus permissions and commenting. Five-plus years of that work is what later let AI plug in like a new car engine.

Asked what excites him about AI over the next decade, Zhao declines a 30-year frame: if AGI arrives, computing itself might not be the point. For this decade, and maybe the next year or two, his sleeper category is RAG and embedding space. Once a language model understands what you put into a computer, you no longer have to organize in order to retrieve. Machines can do that better than anyone. Before, recall meant keyword search, or finding the coworker who remembered the cue. Now you ask Notion AI and get the answer in seconds. Not enough people talk about that, he says. Agents and workflow already have the buzz.

That conviction has staffing consequences. Notion was built by people interested in interface and design: full-stack and front-end heavy, with backend folks for scale, a small search team, and almost no ML. A year of shipping AI taught them that ML people matter. The work is no longer deterministic software you can see working. Zhao does not bake, but it feels like baking: get the ingredients ready, press the button, wait, see if it rises. Sarah’s friends call it probabilistic software engineering. He also offers gardening: lots of preparing and massaging, a different patience and personality.

The other hire is the curious person who learns fast. Prompting turns everyone into a real-time machine-learning engineer: write it right, get the result, then plug the tricks into a user interface. That cohort is now called AI engineers. They tend to be young; Notion has someone under drinking age in the bucket. Researchers are scarce, because the company sits at the application layer: manipulate models, scale to user outcomes, and take tens of millions or a hundred million people—tricky in a way a Twitter demo is not.

The Lego years turned into luck. Because Notion had spent so long on a text editor, it was among the earlier companies to launch AI writing in productivity software at scale. Relational databases were already there, so AI-powered table features followed quickly. A long-built knowledge base made AI Q&A, with a RAG system on top, ship fairly fast. “Just the right moment,” Zhao says. Resourcing it still required conviction. He, co-founder Simon Last, and others live in the Mission; OpenAI did too. Simon’s friends worked there. They visited during the Dota bot days, wondering what the company was doing, and saw early GPTs spit text that was sometimes gibberish and sometimes useful.

Zhao admits he slept on GPT-3. It looked good for marketing, content writing, a creative first draft; it did not click. An early GPT-4 preview did. “This thing can think, it can reason, it can know how to do things, has this little bit of workflow power to it.” That gave him personal conviction that it would change everything. Knowledge work, in his telling, is information-pushing: a piece of paper arrives, a human changes a couple of bits, and pushes it to another human. Language models can now do some of that. After the preview, they bet the company. He compares incumbents to dinosaurs when the asteroid hits: what do you do? They had to figure out which bricks work with AI, hire ML people they did not have, and get the rest of the company moving the same direction.

What is still missing? All of the above, he says: more reasoning, better knowledge, the usual model curve. He treats the technology as a new material. Plastic can do things wood cannot, including bottle water; language models have characteristics deterministic software never had, and every week Twitter finds a new trick. Notion uses the high-end models from OpenAI and Anthropic when it needs reasoning—GPT-4 is smart, Claude 2 is smart—and cheaper, faster models for summarization. The other half of the tradeoff is human. Personal inertia and company risk tolerance evolve slowly. Steve Jobs’s line, as Zhao remembers it, is that you cannot make something too new: stay largely the same and change one or two things. Virgil Abloh of Off-White pushed a 3% difference, new enough to notice and still acceptable. RAG fits that phase: existing behavior, better output.

Zhao’s mental model splits AI into two buckets. One is knowledge and information retrieval—RAG. The other is workflow, where people now say “agent”; calendar sits in that second bucket. He is more excited about the first. Why use a computer? To store and recall. Before language models, recall was keywords, which had to be precise, plus some lexical tricks. With RAG, the model understands what you put in, so you no longer need to organize: throw it in Notion and find it later. For a person or a team that can mean perfect memory, and, if the software is designed right, the right piece of information pushed to the right person at the right time. That is probably more than 50% of knowledge work. The product is still being perfected, and doing it at scale is hard enough that Q&A still had a waiting list.

He has changed how he uses his own company. When he wanted to know how large the move out of the San Francisco office would be, someone had written it in a document. He did not have to ping three or five people. Everyday engineers, designers, and operations staff keep asking one another those questions on Slack or email. Each one, he estimates, is ten minutes to write the answer and twenty minutes to find it, plus delay in the middle. Notion Q&A cuts that to seconds. Elad agrees that RAG and embeddings are under-discussed relative to the impact they are starting to have.

Calendar is the baby step on the workflow side. Why meet, why schedule? To exchange bits between brains. Maybe a language model can do some of that exchange; maybe it can shuffle time. Sarah adds the retrieval half: a calendar that auto-populates what you need to know about the meeting and the people is already valuable. The holy grail is agents doing the knowledge work. The framing that clicks for Sarah is that Notion is eating communication. Why communicate at all? Because the work cannot be done asynchronously, or by the software itself. That is why you talk. Zhao’s guess is that with language models, people will communicate *less*. The agent bet is essentially that the model *is* the communication.

Sarah then asks the uncomfortable product question: if your brain is disorganized, should you still design a tidy knowledge base at input, or dump stream of consciousness? Zhao thinks we may be moving away from the organizational world. Indexes exist because file cabinets needed a way to find things quickly, along certain names or dimensions. Embeddings give a semantic connection among whatever you throw in the bag, and you can pull it out however you like. The experience he wants: see a new idea on a whiteboard, take a picture, dump it, and Notion organizes. First it is perfect memory; later it is a perfect assistant. People love Notion’s left sidebar for organizing a workspace. Maybe the future does not have to have that. He does not want to fall into his own innovator’s dilemma, doubling down on an old paradigm. You do not organize your brain; you dump into it and remember when you wake up. Memory-palace visualization is a minority art. For most people the brain just works.

A friend’s question: what lessons from earlier computing revolutions inform Notion’s AI strategy? A lot is intuition, Zhao says, but history does not repeat itself—it rhymes—and you have to know which phase you are in. There are only two ways to do business: bundling and unbundling. During the recording break he was reading *Romance of the Three Kingdoms*, whose opening line is that the empire, long divided, must unite, and long united, must divide. Business is the same. He thinks we are in a bundling phase.

The 1980s were fragments: WordStar and WordPerfect for text, dBase for databases. A joke he likes: they shipped dBase II first because so many companies had gone bust that starting at II made the product sound like it had been around. The 1990s bundled, because Microsoft had the OS layer. SaaS unbundled again in the mid-to-late 2000s once the web was good enough to run software. Cheap money and easy company creation then produced 10 to 15 years of too many tools and shredded information. Language models, if you build with them, want information in one place and endpoints connected. Talking to scattered endpoints is already hard; the current models do not need it harder. Macro pushes bundling. The technology pushes bundling too.

Elad adds a founder-mindset shift. A decade ago he argued for buying and integrating; in consumer internet, Facebook bundled Instagram and WhatsApp. Then a generation of technical founders insisted every product had to be the best, so they would not build the merely good extra thing. Now companies such as HubSpot and Rippling will ship something 80% as good—or even 50%—and win on the bundle and cross-sell, without every edge-case feature. The YC school of “build one thing, use the internet for distribution” overlapped with the rise of the internet. Another dimension of value is back: it does not have to be 90, if synergy means fewer tabs and a single system of record. Oracle spent a decade and a half buying the second-best product in a category and selling the heck out of it, because deployment was painful and hanging everything on one database had advantages. Zhao draws the distinction: bundling of distribution versus bundling of information. Microsoft and Oracle played distribution. Language models want data in the same embedding space, which is a different kind of value.

Dylan at Figma once heard about a crazy user in the product 14 hours a day. It was Zhao. Guo, an early investor who backed him before Simon Last joined, remembers a design aesthetic that ran from his clothes through the mockups. Zhao’s definition is particular: design is less about how it looks than how the system plugs together. Centralize or decentralize that? Operations-heavy companies can decentralize. Notion is in the bundling business—one information space—so things have to work together. Building it feels like building an operating system or a programming language, and you do not farm a language out to 50 people. Decision-making stays concentrated in the design team: more Apple, less Amazon. The question that interests him is where the Apple for software is. It does not quite exist today.

He studied cognitive science, a bit of philosophy, computer science, art; he learned to code as a kid. The useful trick is stealing from many places. About 80% of Notion’s designers can code, because a designer who codes and an engineer who designs can make tradeoffs other people cannot: squeeze the air bubble toward whichever side is easier. That holism, he says, also keeps the company small relative to its scale. People get to do more than one repeated role. Such people are much harder to find.

The offices match the theory, including the ones that failed. Zhao is Asian; you take shoes off at home. The first office was no-shoes to about ten people, the second to about twenty, the third had heated floors in the Mission. The fourth tried to keep it and failed for a concrete reason: the rug was the wrong rock, handmade, and socks on it hurt. Furniture is supposed to be design classics that last 50-plus years, because chair-makers also made hard tradeoffs, and knowing that history is meant to inspire software built the same way. Guo remembers jazz in the second office and a consistent vibe. Guo also notes they are still early on Enterprise, not at scale, but bundling is already CFO-friendly: fewer tab jumps, and real bills cut for project management and issue tracking.

Asked what lies outside Notion’s scope, Zhao pictures a 1960s office. The notepad on the desk is documents and notes; binders to the left and right are the wiki; the file cabinet behind you is the relational database, with a little cart to put things in. That is the front office, and UI is Notion’s strength. Behind the cabinet is the librarian—the back office, IBM then, Snowflake now—which they do not touch. Back-office cases tend to be vertical, essential, and specific: healthcare, law, specialists who know how the paper gets shuffled and want a language model to help. Front office is open-ended; back office is specific. It will be a race, he thinks, but the market is large enough that it is not necessarily zero-sum. On knowledge bases, RAG changes the game so fundamentally that the left sidebar itself is on the table.