Context Is All You Need: Inside OpenAI with Logan Kilpatrick

Kilpatrick describes the weekend of the board's actions as a very stressful Thanksgiving week, during a break that was meant to let the whole company reset after pushing hard since ChatGPT launched. He says the news on Friday afternoon was shocking because everyone had deep trust in Sam and Greg and the leadership team. OpenAI's culture is transparent, so people usually hear about problems, but this was the first time many heard what was happening between the board and leadership.

What surprised him most was how quickly everyone returned to business. He flew to San Francisco the following week and found people laser-focused on work on Monday morning. He says many companies could have been derailed for a while, and that it speaks to the caliber of the team and its commitment to the mission.

He adds that he is grateful it happened when the stakes were relatively lower than they will be in five or ten years, when AI's impact on work will be larger. He also thinks the event brought the fast-growing team closer, like ChatGPT's launch, GPT-4 and Dev Day had before it.

The most exciting thing Kilpatrick has seen lately is new interfaces for AI, such as the Rabbit R1 hardware device and the infinite-canvas work at tldraw. He finds it striking that chat is still the predominant way people use AI, and imagines a canvas where the AI fills in details with references to files and videos. He thinks 2024 will be the year of multimodal AI and of new UX paradigms.

On where OpenAI will compete, he says the company is focused on general reasoning, coding and writing. Vertical applications like Harvey, which builds custom models and tools for lawyers, are unlikely to be matched by OpenAI's models, since OpenAI's mission is the general case and others can fine-tune and build UI on top.

Developers building general-purpose assistants and agents should not be surprised when OpenAI launches a general agent product, he says, since GPTs already point that way. OpenAI will not launch something like an AI sales agent. To beat ChatGPT as a general assistant, a competitor would need to be radically different, solving ten problems people have with it.

Asked about efficiency, Kilpatrick points to a Harvard Business School study of consultants using AI tools and says more empirical work is coming. As an engineer himself, he ships far faster with ChatGPT and believes engineering is one of the highest-leverage uses, perhaps at least a 50% improvement on lower-hanging tasks. He suggests GitHub's Copilot studies as an analogy.

ChatGPT gives decent answers across use cases but is often not specific to a company's voice or nuance, and GPTs together with Teams and Enterprise plans let companies build domain-specific tools. Lenny shares examples from a friend at Chime, who built a GPT that suggests ads for Facebook and Google and another that explains experiment results like a data scientist.

Kilpatrick has seen GPTs for OKR planning and for venture capital diligence. He uses a private GPT of his own for quarterly planning, loaded with suggestions from an article, which generates a timeline, success metrics and likely cross-functional stakeholders. It forces him to think about things people often miss.

Kilpatrick says prompt engineering exists because models are trained to answer the question you ask, so basic questions get basic answers. He compares it with asking a stranger how their day is going versus asking a friend with shared context. Models are like human-level intelligence with no context about who you are, which is why people who forget to supply it get generic responses.

He expects the problem to be partly solved by the systems themselves, as DALL-E already does: asking for a turtle gets expanded into a rich, high-fidelity description. Text models could do the same, turning a request for a blog post on AI into a detailed brief that the user can edit. Humans are lazy and do not want to type everything they mean, he says.

Lenny describes asking ChatGPT for interview questions in the style of Tyler Cowen. Kilpatrick says the model may know little about a lesser-known guest, so you should supply links to their blog and tweets, perhaps via browsing. The model wants to answer so deeply that it will not tell you it lacks context. He also mentions OpenAI's prompt engineering guide and small tricks like a smiley face or asking the model to take a break, which he says give small gains of perhaps one or two percent.

Kilpatrick explains that sharing a ChatGPT use case used to mean sharing a conversation you had already started. With GPTs, you load the context, files and custom instructions up front, and others chat with a custom version of ChatGPT. Built-in tools include code interpreter, browsing and image generation, and developers can connect external APIs such as Notion or Gmail to take actions.

What excites him most is that non-developers can now solve harder problems by giving the model enough context. At the time all GPTs were free, and he expects monetization in the store later in the quarter, when people get paid based on usage. He also wants simpler one-click sign-in for services like Gmail, since connecting APIs is not easy even for engineers.

His favorite example is Zapier, which lets a GPT reach thousands of connections without code, and Lenny cites a learning GPT called Universal Primer that teaches concepts through questions. Lenny notes that Canva held the top GPT spot at the time of recording.

Compared with older institutions like Apple and NASA, Kilpatrick says OpenAI is young and lacks the accumulated checks and balances that slow things down. The biggest factor, he says, is finding people with high agency who work with urgency. He gives the example of the Assistants API: developers asked for higher-level abstractions, and a group on the team came together bottom-up and built it instead of waiting for a top-down plan.

Planning is guided by principles: whether something helps reach AGI, and, on the developer side, reliability, since nobody cares about a great feature they cannot use reliably. Otherwise the process looks pretty standard, with half-year and quarterly goals. He does not think OpenAI is a big OKR company, and he says revenue is a proxy for compute and better models rather than the goal itself.

He credits a Slack-heavy culture for fast coordination, and says the company was around 780 people at the end of last year and is hiring across engineering and PM. The research team is intentionally small: in a GPU-constrained world, a new researcher who does not level up others can slow everyone else's experiments, unlike adding an engineer to a product team.

Kilpatrick says ChatGPT is moving beyond text in and text out, with voice, image generation and pictures already added. He sees GPTs as the first step toward an agent future where you ask the system to do a task and it comes back when finished. He also sees GPTs as a way to bring the next few hundred million people into AI, since a narrow packaged solution beats a blank slate.

On GPT-5, he notes that GPT-4 was the first model whose capabilities OpenAI could reliably predict beforehand from the compute invested, and it will be interesting to see whether that holds. He expects the usual pattern, where the world first thinks everything is different and then treats the model as an effective tool. The same problems remain, but you have a better tool, and he says unrealistic expectations of GPT-5 doing back flips in the bedroom are wrong.

He adds that planning for people to become used to these tools quickly is an edge. Lenny relays the tip to build toward a GPT-5 future rather than around GPT-4's limits, and Kilpatrick's answer is that better tools mostly help vertical use cases do their jobs more effectively.

For businesses, Kilpatrick says many customers use the API, while ChatGPT is sold as Teams and Enterprise, the latter with things like SSO. The coolest part, he says, is sharing prompt templates and GPTs internally so teams can collaborate on domain-specific tools. Lenny adds that moving employees to a business account with single sign-on can save money and ease administration, and Kilpatrick notes the added control over security and privacy.

The launch the next day included an updated GPT-4 Turbo preview that reduces the model's laziness, and a third-generation embeddings model. He explains embeddings as the technology behind question answering over your own documents, and guesses that Lenny's lennybot.com works by embedding podcasts and posts, then finding the most similar passages to ground an answer.

The new V3 embeddings offer state-of-the-art performance, much better non-English results and are five times cheaper. He recalls that about 62,000 pages of text can be embedded for $1.

Asked where product managers and founders should look, Kilpatrick says products that move beyond the chat interface will have an advantage. He gives the example of a social listening tool where, instead of using dashboards and filters, he wants to ask what people are saying about GPT-4 and get a data-grounded summary. It takes more engineering effort, but that is the unlock that makes a product indispensable.

Lenny recommends Visual Electric, an infinite-canvas image tool for creatives, which his designer wife paid for after trying it. Kilpatrick says this is the moment to build with AI and invites developers to reach out for help getting started.

In the lightning round he recommends The One World Schoolhouse by Sal Khan and Why We Sleep, and shares his life motto, "Measure in hundreds": if you have failed five times, you have really tried zero times. He asks users to send reproducible examples or shared chats when reporting bugs, as with the laziness problem. He closes by saying that people using AI tools, not AI itself, are the ones likely to be more competitive, so this is the best time to learn.