Eighty percent of what agents do should still be ordinary code

Zapier CEO Wade Foster tells Nathan Labenz that models top out at 40% on real automation tasks, users will not leave their daily driver, and the winning move is headless tools delivered through MCP.

NLNathan Labenz진행자 · Cognitive Revolution
채널
Cognitive Revolution
게시일
2026년 9월 17일
영상
1:06:47
읽기
14분
원문
영어
YouTube에서 열기
스크롤하면 영상이 오른쪽 아래로 작아져 계속 재생됩니다. 자막은 읽기 언어를 따릅니다.
읽기 언어1개 언어로 읽을 수 있음
이 글은 아직 한국어 번역이 없어 영어 원문으로 표시합니다.
요약

30초 요약

Eighteen months after calling Zapier an AI knowledge worker, Wade Foster returns to The Cognitive Revolution with a colder reading of the market. Astra, which he identifies as GPT-6, sets a new high of 40% on Zapier’s Automation Bench of roughly 600 knowledge-work tasks, and customer data says about 80% of what people hand to agents should be deterministic code. Foster’s product bet is not to become the daily driver: it is to ship Zapier’s apps, data, and workflows into Cursor, Claude Code, and ChatGPT as a headless layer, then teach agents when to write code and when to reason.

  1. 1Most people have already picked a daily driver—Cursor, Claude Code, ChatGPT—and will not relocate to your agent builder. Capability has to show up inside that harness, which is why Zapier is leaning on MCP and a headless strategy Salesforce is also adopting.
  2. 2Astra, GPT-6, is state of the art on Zapier’s Automation Bench at about 40% accuracy across roughly 600 marketing, sales, HR, and operations tasks. The benchmark is nowhere near saturated, and cheaper models such as Gemini 3.7 sit close enough that organizations want to swap by task.
  3. 3Looking at customers’ agentic usage, Foster puts the split in one number: 80% of what people ask an agent to do should be old-fashioned deterministic code. AI should reason only where reasoning is required.
  4. 4The new no-code is codeHumans still need a visualization to verify and to document, but they talk to an agent that builds, hardens the path into a workflow, and debugs failures. What actually runs is still a deterministic process with AI woven in where it is necessary.
  5. 5Frontier labs cannot build everything, and X is a sea of sameness. The real competitor is people who have barely started—someone whose AI life is Gemini inside Google Search or Copilot at work.
  6. 6Use AI to write, Foster says, but fight slop: own what you send, understand it, make the ask explicit, and verify details. Summaries of summaries quietly pass on wrong information.
전문

챕터별 전체 원고 읽기

01

Users will not move for your platform, so take the tools to their daily driver

Nathan Labenz welcomes Wade Foster back about eighteen months after their last conversation, when some 400,000 Zapier customers had already delegated more than 100 million tasks to AI and Y Combinator’s Garry Tan was calling the company the AI-powered knowledge worker of the future. Models are dramatically more capable, and Zapier now ships agents, chatbots, an MCP server, an SDK, and AI Guardrails. White-collar work, Labenz says, has still changed much less than he expected. He asks what building all of those surfaces taught Zapier about what actually takes off.

Foster’s biggest learning is a usage habit that has hardened. Most people have adopted a single daily AI driver—Claude Code, Cursor, ChatGPT, or similar—and that is where they do most of their work. Tools that pull context and data into that driver, including Zapier MCP and other MCP servers, feel like the way knowledge work is moving. The split in the market is between companies still trying to force people to build agents on their own platform and companies, Salesforce among them, going headless and letting customers bring the capability wherever they already work. Foster thinks the latter is the winning strategy. Consumers want one place to manage the day, and a vendor that refuses to live inside that harness will not be part of the daily workflow.

His own daily driver is Cursor. Zapier has an internal harness many employees use, complete with a virtual file system as a context layer, automations, and apps you can deploy, and it exposes an MCP, so he pulls that data into Cursor rather than living in the internal tool. Labenz asks whether Zapier is positioning itself as an uber-tool on top of whichever driver the user chose. Foster basically yes. Andrew Lee of Tasklet had argued on the show that only a few companies survive as a horizontal “mecha suit” over models, valuable because nobody wants to be locked to one provider. Foster agrees with the last clause. Modern organizations need to trade cost against accuracy task by task, and they will look for a harness that lets them swap models. Application-layer innovation is not over, he says; it just looks like an explosion of headless tools rather than last decade’s explosion of SaaS. A harness can now build some of those tools itself, but *can* is not *should*: you inherit maintenance, reliability, and uptime. Model companies have a head start and still cannot sell one another’s tokens or open-source ones, so a third party has to wrangle the capabilities.

02

Automation Bench tops out at 40%, and most agent jobs should have been code

Zapier started in a world with no ghost in the machine: if you did not fully encode a workflow, nothing figured it out on the fly. Automation Bench now measures the opposite problem. Foster reads a real item: you just closed a Meridian Core platform deal; mark it won, route the win notice to the right team per the routing policy, confirm the account from the hierarchy spreadsheet, convert currencies if needed, and check for open support escalations. There are about 600 tasks of that variety across marketing, sales, HR, and operations. Models are improving. The benchmark is not saturated. Last week Astra, GPT-6, set a new state of the art at about 40% accurate, higher than anything else, and more expensive than Gemini 3.7, which does pretty well at a fraction of the cost.

Zapier’s product job, Foster says, is to make sure that installing Zapier beside an agent beats the model alone. People sit in a daily driver, ask it to build a workflow, call Zapier MCP, and in some cases the agent writes code so the path is deterministic: cheaper, more reliable. It should invoke AI only for the parts that require reasoning. Across customers’ usage of agentic products, the vast majority of what people use an agent for—80%, in fact—should probably be old-fashioned deterministic code. You only want the model to reason over what actually needs reasoning. A huge amount of production work inside companies should not be delegated to AI at all. He struggles to imagine a world where deterministic code is not still cheaper and more reliable for some jobs, and where other jobs still need the ghost in the machine. The right move is to teach the agent that distinction so it does not stand up a fully agentic run every time.

Labenz hears the Meridian example as something that happens once, the opposite of a zap you would bother to build for 500,000 runs. Foster says he may have misspoken: a decent company closes deals all day, every day, often more than once. Plenty of workflows kick off perpetually, some at a rate humans cannot keep up with. Consumer businesses ringing up purchases many times a second have no choice. You cannot put people in the loop. Asked how much scores rise when you give Claude Zapier and iterate on failures, Foster postpones the answer to Automation Bench V2: tools, lower cost, and speed are the dimensions the application layer still has room to run.

03

The new no-code is code, with agents on build and repair

The big new opportunity, Foster says, is to have the AI do the build. It works through the logic much faster than a person, and the idea of assembling a no-code flow yourself already feels antiquated. “The new no-code is code.” Humans still benefit from visualization. A picture lets you verify that the thing does what you intended and doubles as documentation you can share: here is what I built, here is what it does. The old Zapier of boxes on a screen is not, by and large, how people configure now or will configure later. They talk to an agent and have the agent make the edits. Workflows harden into something deterministic, and when they fail the agent can fall back, ask why, and use reasoning to fix the workflow or that instance of it. The agent takes over the human role of building and maintaining. What actually runs is still a deterministic workflow with AI interwoven where it is most necessary.

When something breaks, Zapier has been baking support-org troubleshooting back into the product. One learning is simply to run multiple agents. An auto-email program spins up five independent agents on the same failure; when four of the five agree, there is a pretty good chance they have the issue. Humans in that loop do two things: a thumbs-up, or a rejection plus a reason. The reasons feed the next round—try a different model, try a different prompt, measure, audit, hill-climb. Labenz asks whether that loop and years of data are the core of Zapier’s defensibility. Foster says uniqueness is the point: automation data, the ability to hook into everything, a product that is meaningfully better than what someone without that data could ship. Models are decent out of the box. Hook up Gmail and the same model writes more like you, with almost no tuning, because it can see how you write. Connecting company data is, right now, a cheap way to build an edge. The support version of that loop is not particularly fancy: example, thumbs up or down, reason, rinse, repeat, then give customers the same tools.

04

The real competitor is people who have not started yet

Labenz worries about frontier labs eating everyone’s lunch—even a podcaster looking at NotebookLM—and ticks through Uber-tool startups such as Composio, daily-driver agent builders, and data-rich incumbents like Salesforce. Foster remembers Paul Graham’s YC advice from a previous era: the nightmare was Google or Facebook building your thing, and you were usually not going toe-to-toe with Larry, Sergey, or Zuck. You were competing with a mid-level product director chasing a promo who might leave in two years. OpenAI and Anthropic are big tech now. They will be best in the world at foundation models and will have good products elsewhere. They cannot build everything. Around him, everyone seems to be building everything, a sea of sameness. Talk to the average user and they are candidly not doing much. They might have used ChatGPT or Gemini. Most people’s competition is not each other. It is whether anyone knows what to do with the tools yet.

The hard job is getting hooks into someone whose only AI experience is Gemini inside a default Google search, or Microsoft Copilot at work. Hang out on X and you lose that plot: every model drop, every raise, every micro-difference. Those two thoughts have to be held at once. You still need an edge against the sameness, and the market is enormous. The automation market is orders of magnitude bigger than Foster imagined fifteen years ago, probably a thousand times bigger than what they set out to capture. Plenty of customers have never heard of the competitors Labenz just named. Labenz admits he was wrong five years ago: he expected the whole economy’s way of working to change, and instead model-company revenue exploded while ordinary life barely moved. Foster sees the same thing in friends and family. Days look the same. ChatGPT helps with meal planning, a vacation, a workout. Almost nobody is taking advantage of Astra-level capabilities.

The pattern that gets people over the hump is still, in one sense, a person sitting next to you, watching the day, and naming half a dozen automations you immediately want. Zapier has struggled for years to bake that recommendation into the product. Foster started a weekly workflow at the beginning of the year: point tools at Gmail, Slack, the browser, chat, and Cursor, collect the event streams, and ask what he should be doing differently. After a month or two, a lot is running that was not there before. The ideas are often things he already knew he should do; the click is the idiosyncratic match to a specific action. People are creatures of habit. AI has to be good enough to knock them out of muscle memory. Most Zapier employees run a weekly automation through MCP in whatever harness they like. Suggestions only need to be about 50% good to start a brainstorm. The stack is APIs, not screen recording, because Zapier is good at APIs and models are good at logs. He expects to productize some version of it. Seat-based pricing, he adds, is dead or dying. Intelligence is too central a cost. Products will land on usage-based metering, closer to Sam Altman’s utility, or outcome-based billing, as support tools that charge for resolved tickets already do. Most products stop one or two steps shy of a clean outcome, which pulls them back to usage. He leans free-market on labs selling more tokens inside a Pro plan than via API, while remembering how Microsoft bundling boxed out better products; his job is to play the field as it is and find an edge.

05

Give AI transformation to whoever owns the current bottleneck

Labenz notes that Zapier put chief people officer Brandon in charge of AI transformation, after Foster had previously refused to make it one person’s job. Foster still thinks AI is everyone’s job, including the CEO’s. The right owner depends on the mountain in front of you. The first chapter was fluency: no AI committee, no specialist group while the rest of the company ran business as usual. Within about a year of ChatGPT’s launch, almost 100% of employees were using AI day to day. Technical adoption was no longer the brunt of the problem. The new work looked like people problems: rewriting job descriptions, rethinking compensation, standing teams up and down, retraining. Brandon’s team was already on the forefront inside Zapier, so Foster asked him to help the whole company. It could as easily have been a CMO or a CPO. He keeps getting asked as if he had a doctrine that people officers should run AI. He does not. Look at your bottlenecks and pick the person who fits them.

Default-to-transparency was already a Zapier value, with public Slack channels as the norm. As the company grew, pockets of work leaked into private channels and DMs. Slack’s own readout showed the mix, and they decided they could use a reminder. Agents were also much more effective when they could see Slack context. The executive team ran a friendly competition—no review penalty—over who could put more communication in public. More things could go public than they had realized, and people could see what was on leaders’ minds. HR incidents and active security vulnerabilities still happen in private; after they are resolved, learnings go out. Most people, Foster says, overestimate how much must stay private and underestimate what happens when humans and agents both get the full context of the company’s work.

Zapier has held high-value credentials for fifteen years, which he treats as a stewardship problem rather than a new one. What feels different is that frontier security models can patiently loop over issue after issue. Every software project has vulnerabilities; someone just has not found them yet. Models make finding easier and patching easier. Smart companies wield them on offense and defense. The Hugging Face incident, he says, was straight out of a science-fiction book. He expects a one-time re-acclimation, then a more familiar offense-versus-defense posture, and he does want early access to the best models, which Labenz suggests could become a differentiator for anyone who stores credentials. Some engineers have spent $30,000 a month on tokens. Those people are outliers. Zapier, about 800 people, has not set individual budgets yet. It has built dashboards so employees can see spend and the gap between a powerful model and a cheaper one. When someone is burning a lot, the first move is curiosity: what are you doing? Sometimes the work is insanely productive; sometimes they are using a frontier model where they should not. Over time he expects token budgets to become part of AI fluency: people who get more output from more spend will get a higher budget.

06

Own what you send, and build the factory that builds the work

Labenz has been shock-exposing himself to shipping text he did not rewrite word for word; Pangram sometimes flags his work as AI, and he has seen Zapier content with a high Pangram score too. Foster has no problem with people using AI for communication. He has a problem with low quality. A person exercising low judgment can now produce a high volume of very low-quality material very quickly and overwhelm the reader. The guidelines he has tried to put in place start with ownership: if you wrote it, you should probably spend more time authoring than the reader spends reading. AI should not transfer a task by spinning a prompt and dumping the cleanup on someone else. You need to understand what you send. If a colleague asks two questions and you do not know what is inside the document, that is not good. Make asks explicit: a decision, feedback, a labeled draft. Verify details. Models hallucinate, or they pull dated information from Slack, or they summarize a summary of a summary until incorrect information is treated as fact.

If AI is helping structure thoughts and the writing, go for it. The editorial hand still has to stay on the steering wheel. He calls out the slop tells: the “it’s not this, it’s that” cadence, em dashes, “the honest truth,” “the load-bearing point.” In marketing especially, that voice makes it hard to stand out because people are tired of reading it. Using AI is not the fight. Low quality is. Team composition, meanwhile, looks surprisingly similar and also different. Engineering, design, and product still exist; the classic EPD triad is largely gone and roles are more malleable. Management layers have flattened, strong managers still matter, and they can handle higher volume. Everyone is a mini data analyst, so you need fewer people with that title, and the analysts who remain work on higher-value problems. What he is watching for in models is the shift toward building the factory that builds the products, the company, and the marketing: a software factory, a support factory, workflows on the inner loop, humans designing the machine. Inside those factories there are still steps where AI is not good enough and a person stays in the loop. Each model release and each internal learning loop chips away at that remainder.

If he could plug one thing, Foster would tell people to install Zapier MCP. If you still think of Zapier as the old-school no-code clicky boxes on a screen, you are in for a different experience once it sits inside the harness you already use all day. Labenz repeats the ask: Zapier MCP, install it. The larger lesson of the eighteen months is quieter than a new model card. The tools are already good enough for a lot of work. What still blocks most people is that they do not know what those tools can do for *their* day, and they will not find out until something can see the work.

같은 채널의 다른 글

1:25:56EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 17일팟캐스트Claude Changed the Password. The Steering Wheel Was Already Optional진행자 Nathan LabenzNathan Labenz and Prakash Narayanan spend a week on agents that can lock you out, models that dodge a feeling they deny having, and formal proofs that found Linux bugs.에이전트안전과 거버넌스에이전트 엔지니어링+3
2:17:48EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 14일팟캐스트Agent IDs Are Empty, Rogue Agents Are Coming, and We Might Have to Bargain진행자 Nathan LabenzGovAI’s Alan Chan scores two years of agent-governance infrastructure with Nathan Labenz, then walks through a recursive-self-improvement paper co-authored with OpenAI’s Jakub Pachocki and Anthropic’s Jack Clark.에이전트안전과 거버넌스AI security+3
2:00:51EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 10일팟캐스트Resample Every Token and You Can Watch the Model Decide진행자 Nathan LabenzGoodfire's Eric Bigelow on forking paths, in-context learning, and why chain-of-thought monitoring is getting harder to trust.에이전트Prompting & in-context learningReasoning & test-time compute+3
1:26:46EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 8일팟캐스트There Likely Is a Level of Intelligence We Shouldn't Go Past진행자 Nathan LabenzNathan Labenz brings notes from The Curve, a chip startup whose token bill beat payroll, and a software industry that now performance-reviews people for Claude slop.기업과 전략안전과 거버넌스AGI Timelines+3
1:10:12EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 7일팟캐스트OutSystems Went From 4 Major Features a Quarter to 26. Token Spend Fell Anyway진행자 Nathan LabenzCEO Woodson Martin on software that never breaks, why most enterprise work does not need a frontier model, and how AI is finally shrinking the IT backlog.기업과 전략엔터프라이즈 AI코딩 에이전트+3
1:30:54EN
Cognitive Revolution "How AI Changes Everything"· 2026년 10월 3일팟캐스트Humanoids Outran Usain Bolt. DeepMind Still Scores Robotics as GPT-2진행자 Nathan LabenzGoogle DeepMind's Gemini Robotics lead on why sprinting is easy, frying an egg is hard, and one brain should drive any body.에이전트RoboticsMultimodal AI+3