How Linear Built an AI Agent for Real Product Work

Linear's Nan Yu and engineer Jacob Shumway discuss how they built a production AI agent with Peter Yang. Their examples cover a prototype, dynamic skills, evaluations and the places where an agent enters an existing workflow. The episode is sponsored by Oceans.

Shumway describes an agent as a model operating in a loop: it works toward a goal, calls tools, gathers context and eventually returns a result. Yu adds that an agent product contains more than the loop, including interfaces, scheduling and other cooperating subsystems.

Yu shows an early 2025 memo about automating routine program-management work. The team's ambition later expanded to interactive and creative tasks. An early prototype called a model from the frontend and exposed existing command-menu actions as tools; creating issues from notes became an especially useful starting point.

The first production version was quietly connected to Linear's existing Slack integration. People discovered they could mention the agent in a conversation and ask it to act on the surrounding discussion. The team learned from the unexpected ways people used it rather than assuming every use case in advance.

Giving the agent all possible actions or an entire GraphQL schema created context and hallucination problems, Shumway says. The team moved toward skills loaded on demand: a request determines which instructions and tools are brought into context, including product-specific guidance for writing and prioritizing issues.

In one demonstration, a Slack discussion leads to an issue and then a pull request. The agent delegates work to itself, while the issue remains assigned to a person and tracked in a backlog. The guests describe the result as a starting point for a human to inspect, not a guarantee that the generated change is right.

Early usage centered on creating issues, so the team used a small router and an optimized path for that common task. Shumway says they generally start with a strong model to establish that the workflow works, build evaluations and define success, then consider smaller models to reduce cost while checking quality.

Their evaluation dataset grows from real usage and unexpected behavior. Deterministic tests check concrete requirements such as a requested status; an LLM judge can assess more subjective questions such as title selection. Shumway says they use model judges less often because the judges themselves need checking, and unnecessary consistency checks can generate false signals.

The team also tests interaction quality: did the agent recognize an actual request, or act too eagerly? Feedback about awkward behavior becomes additional evaluation cases. Shumway recommends a clear goal and tools to retrieve relevant guidance instead of filling every prompt with potentially distracting instructions.

When an agent lacks a tool for a requested action, it can report that gap. The team routes the report to an existing issue or creates a new one. The guests say users also found unexpected applications, including translating customer feedback before filing an issue.

Yu says a native agent lets Linear apply its product-management opinions across the broader workflow, while external agents can also connect to individual tools. He argues that builders should map the user's real multi-step process and identify entry points such as Slack discussions and meeting debriefs, not assume a chat box is the whole product.

For release, Shumway says the team chose to ship once the agent was good enough to learn from more users. Yu describes the product bar as reliability on the core use cases they actively promote, while acknowledging that less common tasks can vary in reliability.

The guests identify proactive behavior and longer-running project memory as areas they were working toward. Those are plans discussed in the interview rather than a promise of current behavior. Their recurring lesson is to start with a useful workflow, observe real use and improve the agent with targeted context and feedback.