Harrison Chase on Planning, Interface Design and Memory for AI Agents

Matthew Berman introduces a talk by LangChain co-founder and CEO Harrison Chase at a Sequoia event, pausing throughout to add his own commentary. The May 2024 discussion focuses on planning, user experience and memory for AI agents. Berman discloses in the video description that he is an investor in CrewAI, a framework he praises during the discussion.

Chase describes an agent at a high level as a language model interacting with the external world. Berman expands on a diagram showing tools such as calendars, search and code execution, together with memory, planning and actions. The model is one component of a larger system rather than the entire application.

Berman distinguishes short-term conversational context from longer-term stored information, such as material retrieved later through RAG. He says frameworks can add planning methods such as reflection, self-critique and breaking a task into subgoals. His claims about a particular framework's performance are his own assessment, not a controlled comparison in this talk.

Chase gives a simple mental model: run the language model in a loop, execute the action it chooses, report the result and ask what to do next until it finishes. His interest is how developers turn that idea into reliable, production-ready applications, particularly through planning, interface design and memory.

Berman explains reflection as generating an initial answer and asking the model what could be improved, and points to Tree of Thoughts as another approach to planning. He speculates about future models and rumored systems that might plan more effectively, while noting he has not established such capabilities in the example he mentions.

Chase says that, at the time of the talk, models were not reliable enough to choose and execute every next step unaided. Developers therefore added explicit planning at the beginning or reflection at the end. He asks whether these prompting and orchestration techniques will remain necessary or become part of model APIs.

Berman guesses that stronger reasoning may require an architecture beyond transformers, but presents that as a guess. He argues that even if models improve, frameworks could still coordinate models, tools and agents. Chase explicitly says he has questions rather than settled answers about which planning techniques will endure.

Chase discusses “flow engineering,” citing AlphaCodium as an example of improving a coding workflow through an explicitly designed graph or state machine. In that framing, human engineers make some planning decisions in advance rather than asking the model to decide every step. Berman sees workflow design as a separate area of experimentation beyond writing a prompt.

For agent user experience, Chase says human involvement remains important because agents are not sufficiently reliable. Berman agrees that businesses care about consistency and quality, but stresses a trade-off: too much intervention removes the value of automation. He suggests reviewing substantial deliverables while experimenting with where human oversight is most useful.

Chase praises a rewind-and-edit interaction demonstrated in Devin: a user can return to an earlier point, change the agent's state or work, and steer it from there. He sees this as a way to make agents easier to guide. Berman also likes the interface that shows chat, browser, terminal and code together, while acknowledging debate about what the original demo actually achieved.

Berman points to Pythagora as another example of returning to an earlier project step and continuing after an edit. He discloses that it had previously sponsored his channel. His enthusiasm for this interface is a product judgment, not proof that rewinding alone ensures reliable results.

Chase turns to memory with a demonstration in which a user corrects an assistant's writing style through conversation. After feedback, the application recalls the preferred style on a later visit and allows further refinement. Chase calls this procedural memory: remembering how a user wants a task done.

Chase distinguishes procedural memory from personalized facts. In a journaling example, the application remembers a user's interest in Italian food after a cooking-class conversation. He expects both kinds of memory to matter for future agents, though the examples are demonstrations rather than evidence of perfect recall.

Berman connects memory to business applications: an agent can retrieve organizational knowledge and retain useful context for future work. He also asks practical questions about what to store, when to forget and how to update memories when a business changes. He concludes that the right mix of tools, models, memory and oversight remained an open design problem in 2024.