Bilt Saved 65 Support Seats. Decagon's Founder Says Most Agent Ideas Still Won't Sell

Jesse Zhang on why customer support is the golden agent use case, why instruction following beats o1-style reasoning, and the two filters that kill most agent startups.

SGSarah GuoModeration · Conviction
Kanal
No Priors
Veröffentlicht
16. Jan. 2025
Video
30:09
Lesezeit
10 Min.
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 1 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

On No Priors, Decagon cofounder and CEO Jesse Zhang tells Elad Gil why customer support is tailor-made for today's language models: you can roll the agent out on 1% of traffic, and the ROI shows up in headcount and CSAT. A Bilt Rewards case study put the savings at about 65 agents after a year; Klarna's public numbers gave the whole category a tailwind. Zhang is still bearish on most other agent products in the near term, because they either have to be perfect on day one or cannot put a dollar figure on the copilot.

  1. 1Customer support is the golden use case for AI agents because the shape of the work matches what language models are already good at, and almost any company with a sizable support team is a fit.
  2. 2Buyers, Zhang says, really watch two numbers: what fraction of conversations the agent can handle alone, and whether CSAT or NPS went up. Accuracy is next, and it is nearly non-negotiable in regulated industries.
  3. 3Bilt Rewards stopped scaling support about a month in and, after a year, quantified savings at around 65 agents — plus social posts from users who said it did not feel like any chatbot they had used.
  4. 4Everyone has the same modelsDecagon treats itself as a software company: the alpha is orchestration around the customer's business logic, plus classic tooling for transparency, observability, and control.
  5. 5o1-style gains are mostly quantitative reasoning. Support lives or dies on instruction following — executing an SOP to the letter — and that curve, Zhang argues, has not plateaued.
  6. 6Agent products that work share two traits: you can roll them out slowly without being perfect, and the ROI is a one-line calculation. Security and text-to-SQL fail those tests; support and coding agents pass.
TRANSKRIPT

Das vollständige Transkript nach Kapiteln

01

Customer support is the golden agent use case because it fits what language models already do

Jesse Zhang grew up in Boulder, did math contests as a kid, studied computer science at Harvard, and started a company right out of school. That first company was eventually bought by Niantic. He left to cofound Decagon with Ashwin; they met through mutual friends and officially at a VC offsite. The big lesson from the first company, Zhang tells Elad Gil on *No Priors*, was not to overthink. They were excited about AI agents — arguably the coolest technology of their generation — and started by talking to customers rather than writing a vision deck.

That customer tour is how they landed on customer interactions and customer service as what Zhang now calls the golden use case. The work is tailor-made for what language models are good at. They still were not thinking much about vision; they had customers in front of them and wanted those customers to be happy. Decagon, founded in August 2023, now ships AI agents for customer service and customer experience. The platform is already in production at large enterprises and fast-growing startups including Rippling, Notion, Duolingo, ClassPass, Eventbrite, Vanta, and Substack.

What has made the company special so far, Zhang says, is a huge focus on transparency. Larger buyers will not accept a black box. They need to see how decisions are being made, what data is being used, how answers are produced, and they need a way to give feedback. Pretty much any company with a sizable support operation is a fit.

02

After Klarna's tweet, buyers really only watch two numbers

Gil opens the business discussion with Klarna. The European buy-now-pay-later company's CEO posted that in the first four weeks, AI support handled 2.3 million chats, matched human customer satisfaction, cut repeat inquiries 25%, and resolved issues in two minutes versus eleven for a human. The system was live 24/7 in 23 markets and 35 languages, and about 700 full-time agents were shifted to other work. Zhang calls the article awesome tailwinds for the whole industry. Evangelists are shipping the same use case, and it is no longer much of a hot take to say these agents will be pervasive in a couple of years.

The benefits land in the same vein, but different buyers prioritize different things. For Decagon's customers the leaders, by far, are two questions. What fraction of total work — in this case conversations — can the AI agent do, and how much work is that saving? And how much happier are customers: satisfaction score, NPS. Accuracy sits behind those two, and in a regulated industry it has to be very high. The economics are not subtle: you save money, time, and headcount, and on the other side you make customers happier, which can mean higher retention and more conversions.

Zhang's one-line pitch is that you are giving every customer a personal concierge in their pocket, in any language, 24/7. That, he says, can be pretty transformational. The Klarna numbers made the category loud. The scoreboard Zhang actually sells against is still those two metrics, plus accuracy for anyone who cannot afford a wrong answer.

03

Bilt Rewards stopped scaling support in a month and later counted 65 seats saved

The case study Zhang can talk about on the record is Bilt Rewards, the rent-and-points platform. It has a large, fast-growing user base; friends of his use it to earn points or make payments. Support inquiries grow linearly with users, so exponential user growth means exponential ticket growth. When Bilt first came in, the goal was blunt: they were getting overwhelmed, and they wanted to know whether AI could help.

Within about a month of going live, Bilt was able to stop scaling the support team. The agent took over a lot of the automation and smoothed the operation. Almost a year in, they had restructured the team and quantified the savings: around 65 agents of headcount. That is a very easy ROI conversation. The softer win, Zhang says, is snappier experience — and social posts from users who tried Bilt support and said it did not feel like any AI or chatbot system they had used before.

04

Everyone has GPT-4 and Claude. The alpha is orchestration and the software around it

Gil asks the industry's standard question: GPT-4, GPT-4o, Claude Sonnet — anyone can call those. What has Decagon had to build on top? Zhang's answer is blunt. Everyone has the same access to the same models. Decagon sees itself very much as a software company. AI models are tools everyone can use. Most of the alpha, or the special stuff, is on top of the models: either an orchestration layer, or the software around it. They have focused hard on both.

Orchestration is how you use different models together. You stand up evals that measure how good each model is at certain things, put them together, and mold the pipeline around the customer's business logic. That layer will look different for a support agent than for a coding agent, but at the end of the day you are building structure on top of the LLMs. The other half is classic software aimed at not feeling like a black box: what data the agent used, what steps it took, whether you can analyze the conversations coming in.

If a company gets a million conversations a year, no one is reading them. Decagon lets the LLM read every thread, report how things are going, find gaps in the knowledge base, and break out the big categories leadership should care about. Gil later says customers tell him three things: Decagon ships fast, the team is responsive, and — most important — the product tends to outperform. Zhang attributes the last of those to transparency. In a market this quantitative, buyers put you in production on 1% of volume, ramp from there, and benchmark you against Salesforce and other alternatives. Observability, explainability, and control are how you win that bake-off, and he thinks there is still a long way to go.

05

Support needs instruction following more than o1, and voice is still a latency problem

Gil notes that true agentic systems — sequencing events, chaining reasoning — still look early, even with o1. Zhang splits intelligence into types. A lot of recent progress from o1 and Sonnet has been quantitative reasoning: better at coding, better at math. Those help Decagon, but they are not the biggest difference-maker. In support, the intelligence that matters most is instruction following: you have a pile of rules, and the model has to follow them to a tee. When people say the core models have plateaued, he thinks they mean reasoning. Instruction following for an SOP or a workflow is still a live research problem in the major labs, and it is what he is looking forward to next year.

Voice is the other under-discussed piece. Gil argues the industry talks endlessly about text understanding and underplays text-to-speech and speech recognition, with Cartesia, ElevenLabs, OpenAI, and Google all offering APIs. Zhang says the impact is huge. Customers do not care about channel: some prefer voice, some chat, some email, some SMS, and Decagon's job is to handle all of them. They started with text because it is easier to build and easier for a customer to evaluate. Big companies that have seen the text-agent scorecard are now asking for the same thing on phone calls. None of that would be possible without those voice models, and realism and latency have both jumped this year.

Latency is still a big deal. A caller expects the conversation to feel snappy; the classic pipeline of transcribe, run the model, then synthesize adds pauses. Voice-to-voice models, including work from OpenAI, have great latency, but production often needs extra compute — fetching data, multiple model calls — so end-to-end audio is not always enough. The staged pipeline can cheat with streaming: start generating the first words, or have the agent say it needs a second to look up the account. Each customer has different trade-offs, and Decagon is working closely with the voice-model companies to tune those choices at scale.

06

The math-olympiad pipeline into AI startups was always there. Entrepreneurship is what changed

Gil counts the pattern: Ramp, Braintrust, Pika, Cognition (the company behind Devin), and Decagon were all founded by people with math olympiad or informatics-olympiad backgrounds, around the same age, who have known each other since middle and high school. Decagon has even hosted math-olympiad events as team gatherings. Zhang's explanation is unromantic. The community was always there. Math contests have been around a long time, and they are a way for very sharp kids to meet. What changed in the last five or six years is that startups became a mainstream destination. That same demographic used to flow into academia and quant trading. Now there is an influx of those people into company-building, and they can see what their friends are shipping.

The help is informal. Zhang has angel-invested in a lot of the companies Gil listed; a lot of those founders are angels in Decagon. Company-building has infinite surface area — hiring, sales, compensation, how you actually build the thing — so extra data points are useful. They hang out, play card games, a Chinese version of bridge. On hiring, the shared contest background is decent signal, and cofounder Ashwin did similar contests growing up in India. For the most part San Francisco is full of talented people with or without olympiad medals, and the hiring process is more or less the same. The events are the nicer trigger: puzzles, friends of friends, a community that likes the same things.

07

Most agent ideas fail one of two tests: you cannot ramp them, or you cannot price them

Looking out 12 to 24 months, Zhang is excited about models getting better across modalities, not just voice. Many Decagon customers have software products; an agent that could see the user's entire screen and then help navigate would be a large jump. Anthropic's computer-use demo points in the right direction, but in his opinion it is not production-ready yet. The broader thesis is that an explosion of agents across use cases is now hard to deny. Some categories will take longer than others. The nature of human work will change with them: a lot more people will be supervising and editing agents. That is a 2025 investment area for Decagon, because monitoring an infinitely scalable system you can sometimes hard-code is a different job from giving a person real-time feedback.

The question they obsessed over at the start was why some agent scenes roll out fast and others stay as demos. In hindsight, the use cases that emerged share two qualities. First, you have to be able to roll the thing out slowly. It does not have to be perfect off the bat, and it already has to provide value. Security looks like a perfect hunting ground — tons of logs, structured data — but the job is to catch any small thing, and models are inherently non-deterministic. Buyers will not trust an agentic solution there. Zhang thinks enterprise adoption in that category will be a lot slower than the demos imply.

Second, the ROI has to be easy to quantify. Text-to-SQL and the so-called AI data scientist look useful until everyone realizes someone still has to monitor and edit, so the product becomes a copilot. Most teams do not have that many data scientists anyway; benchmark the agent against a real one and it will not replace them, which makes a large contract hard to justify. Support passes both tests: you can put 1% of traffic in production, and teams already track satisfaction, resolution, and headcount. Coding agents work the same way — section off some tasks. Zhang's near-term takeaway is that he is probably more bearish on a lot of AI-agent use cases until the models unlock the next batch.

Mehr aus diesem Kanal

39:23EN
No Priors: AI, Machine Learning, Tech, & Startups· 15. Okt. 2026Podcast · 2 StimmenThe Defenders' Window: Greg Brockman on AI-Generated Knowledge, Drug Discovery, and the New Physics of CybersecurityModeration: Sarah Guo, Elad GilOpenAI's president explains why solving open math problems is a proof point for science, how proactive agents change product form, and why security spending must exceed software spending.SicherheitAI securitySicherheit & Governance+3
1:09:44EN
No Priors: AI, Machine Learning, Tech, & Startups· 9. Okt. 2026Podcast · 2 Stimmen10,000 GB300s for Four Weeks: Reflection’s Open-Model LedgerModeration: Sarah Guo, Elad GilMisha Laskin tells No Priors how Reflection trained Beam, why reinforcement learning outspent pre-training, and why the West needs its own open weights.Open Source & CommunityOpen-weight modelsReinforcement Learning+3
35:39EN
No Priors: AI, Machine Learning, Tech, & Startups· 2. Okt. 2026Podcast · 2 StimmenCompute Scaled a Millionfold. Bandwidth Only 40x. Fractile Wants to Fix That RatioModeration: Sarah Guo, Elad GilIn the summer of 2022, Walter Goodwin founded Fractile in London to build very fast inference chips for the world's largest models.Infrastruktur & DeploymentInferenceAI cost & performance+3
42:44EN
No Priors: AI, Machine Learning, Tech, & Startups· 24. Sept. 2026Podcast · 2 StimmenThey Paid $7.7 Billion for a Broker — and Intend to Do One Deal a YearModeration: Sarah Guo, Elad GilSequence Holdings co-founder Michael Lee tells No Priors why owning a market leader, not selling it software, is how AI actually remakes an incumbent.Unternehmen & StrategieEnterprise-KIAI Strategy+3
38:14EN
No Priors: AI, Machine Learning, Tech, & Startups· 18. Sept. 2026Podcast · 2 StimmenDiffusion Will Win Inference: Stefano Ermon Bets Parallel Tokens Beat Custom SiliconModeration: Sarah Guo, Elad GilThe Stanford diffusion pioneer tells No Priors why a 50-person lab is training language models that generate many tokens at once — and why that maps to GPUs the way transformers once mapped to training.Modelle & TrainingInferenceAI cost & performance+3
45:10EN
No Priors: AI, Machine Learning, Tech, & Startups· 10. Sept. 2026Podcast · 2 Stimmen76% of Agentic Commerce Is Under 30 Cents — So Coinbase Is Banking the AIsModeration: Sarah Guo, Elad GilBrian Armstrong tells Elad Gil how Coinbase became an everything exchange, why card networks cannot price agent-to-agent payments, and what his longevity company is taking into the clinic.AgentenAgentic commerceKI im Finanzwesen+3