Cynthia Bailey: Choose AI models for the task and guard your API keys

Stanford’s Cynthia Bailey shows how token overhead, repeated agent steps and exposed credentials turn AI use into real costs.

CBCynthia BaileyIntervenant · Stanford University
Chaîne
Stanford Online
Publié
19 nov. 2026
Vidéo
48:59
Lecture
7 min
Original
anglais
Ouvrir sur YouTube
En faisant défiler, la vidéo se réduit en bas à droite et continue. Les sous-titres suivent votre langue de lecture.
Langue de lectureDisponible en 2 langues
Cet article n’est pas encore disponible en français ; il s’affiche dans l’original en anglais.
RÉSUMÉ

En 30 secondes

A larger model can make a multi-agent tool slower and more expensive without improving a simple check, says Stanford lecturer Cynthia Bailey. She compares an objective tutoring guardrail with a policy verifier that needs more nuanced judgment, and shows how cached context and unused skill listings affect token costs. Her live demonstration also shows why an API key embedded in a web page can be copied by anyone. She closes by connecting AI use to the water, electricity and community impacts of data centers.

  1. 1Reusing repeated context can reduce input costsBailey describes cached attention work and says cache reads can cost up to about 90% less than uncached input.
  2. 2Unused skill listings can add tokens to every promptIn her usage-report example, four skills contribute 225 tokens of context; she recommends removing ones the agent does not need.
  3. 3Estimates of AI’s resource use depend on the taskBailey cites Google’s 2025 median Gemini estimate and larger multipliers for image, video and reasoning, while noting limited public data and uncertain comparisons.
  4. 4A guardrail that retries answers can multiply latencyStanford’s tutor checks generated answers and may send them back several times before returning one to the student.
  5. 5Two output checks can need very different modelsBailey contrasts a straightforward code-length rule with checking whether cited policy actually supports an answer.
  6. 6A key in public website code can be reused by anyoneHer deliberately invalid-key demo shows how viewing page source exposes a credential embedded in a web app.

À retenir

  1. Choose a model for the difficulty of each step; repeated agent calls multiply latency.
  2. Inspect input tokens from context and skill listings, and start fresh when switching tasks.
  3. Use prompt caching when repeated context is available, while checking provider pricing.
  4. Treat an API key like a password and keep it out of code sent to website visitors.
  5. Resource estimates vary by model and task; Bailey cautions against treating proprietary-model costs as precisely known.
TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Transcription seule
01

What a usage report counts

The report separates session time, model-response time and token usage. Bailey explains how input, output and cached context contribute.

As projects grow, token usage becomes a practical cost to monitor. I will also consider electricity and water, the choice of model for an agent workflow, and how programs use API keys.

Tools for comparing models can help you swap providers and manage usage. Claude Code’s usage report offers a concrete place to see what a project consumes.

A sample usage report shows a total cost of $28 and two measures of duration. The API duration adds up the time waiting for responses across the session, including processing, network transit, and congestion at the data center.

Wall-clock duration is the time elapsed since the session began, as measured by a clock on the wall. It differs from time the model or processor spent handling requests.

Input tokens count everything the model reads, not just the prompt you typed. Settings, system instructions and enabled tool connectors can also occupy the context window.

Output tokens include generated answers and, for reasoning models, generated thinking as well. The underlying process still produces tokens, even when the user sees only a final response or a summary.

Caching can reduce the cost of repeated input. Much of a session’s context, such as the system prompt and settings, stays the same from one request to the next, allowing precomputed work to be reused.

Bailey says cached reads can be discounted by up to roughly 90% compared with uncached input. She relates the saving to storing previously calculated attention work instead of recalculating it for repeated context.

The attention mechanism weights words according to context: after ‘the teacher said welcome, please have,’ it raises the likelihood of ‘a seat’ relative to unrelated alternatives. Its matrix calculations cost resources, which is why reuse helps.

02

Unused skills still take context space

Skill listings add overhead to prompts. Removing unneeded items and changing sessions with the task can limit it.

A usage report can also show the footprint of plugins and skills. In the example, one category containing four skills contributes 225 tokens to the context for each prompt.

Even when you do not invoke a listed skill during a session, its listing can add overhead. Bailey recommends removing unneeded skills, inspecting the context, and starting a fresh session when switching tasks.

The report also suggests assigning cheaper models to general-purpose subagents. Model choice changes both the dollar cost and the time a workflow takes.

03

Why water and electricity estimates are uncertain

Public data is limited and different tasks use different amounts of compute. Bailey compares reported text estimates with image, video and reasoning multipliers.

Token counts and prices are only part of the cost. The electricity, water and carbon associated with model use matter too, although limited public data makes precise comparisons difficult.

Bailey cites Google’s self-reported 2025 estimate for a median Gemini text prompt: 0.26 milliliters of water and 0.24 watt-hours of electricity. What counts as a median prompt is not very clear, and newer models or reasoning patterns may have different costs.

She cites estimates that image generation can use 28–40 times as much energy as text and video 1,000–3,000 times as much. Such comparisons depend on the task and the models studied.

Research on open models also found substantially higher costs for reasoning modes, with an average around 30 times and much larger ratios for some models. Bailey cautions that the wide variation makes it difficult to apply a single multiplier to proprietary models.

Long contexts add to processing demand as well. Bailey points to Epoch AI calculations comparing the cost of contexts with 10,000 and 100,000 input tokens, and again suggests starting a new chat for a new topic.

04

Match each agent to a model

Latency adds up across a workflow. A tutor and a policy chatbot show why simple and nuanced guardrails can need different models.

A larger model is not automatically the best user experience. Greater capability can come with latency: the time a request takes to produce a result.

Latency compounds in a multi-step agent workflow. A delay incurred repeatedly as work passes between agents can matter much more than the latency of a single interactive exchange.

Stanford’s Wayfinder tutoring system provides an example. Its goal is to help computer science students learn rather than simply produce their assignment answers.

An input guardrail first checks whether the student is plainly asking the system to solve the assignment. If so, a prepared response encourages a conceptual question instead.

A response agent then produces an explanation, drawing on examples of helpful course-staff responses and citing class materials. It might use an analogy or begin a Socratic dialogue.

An output guardrail checks whether that response still gives away too much. This second check matters because a student might phrase a request in a way that gets past the first filter.

If the output check rejects a response, it sends it back for another attempt. Bailey says testing has sometimes required several iterations, multiplying the effect of a slow model.

The class exercise is to choose models for each agent, considering model generation, size, capability, speed and price. The agents do not all need to use the same model.

A policy question-answering system illustrates a different verification problem. Stanford has a chatbot that searches staff and faculty handbooks for matters such as research compliance or leave procedures.

A policy answer needs a check that citations exist and that the cited text actually supports the response. A real citation can still be misinterpreted; an invented one can look convincing.

Bailey contrasts that nuanced check with the tutoring guardrail’s more objective rule: a response should not contain more than three consecutive lines of code. She expects a small, inexpensive model may be appropriate for that narrower check.

The policy guardrail has responsibility comparable to the response generator because it must interpret context, policy and citations. Bailey would choose a different model for it even though both components are called output guardrails.

05

API keys allocate both access and cost

A key lets software call a model on an account. Builders can pay centrally or ask users to supply their own credentials.

An API key lets a program use a model outside the provider’s chat interface. It acts like an account credential for software, identifying whose account should be charged for requests.

Because the key can authorize use billed to its owner, it deserves the same protection as a password. Do not leave it in a public place or casually transmit it to others.

A service builder can choose who pays. For a tutoring system serving an initially 550-student class, Bailey chose to supply her own key so students could use the tool freely and equally for classwork.

Another design would have each student register for a key and pay for their own usage. That distributes costs according to use, but requires each user to set up an account and payment method.

06

A web page can accidentally expose a key

A deliberately broken translator demonstrates a credential visible in page source. Bailey asks builders to keep keys out of public code.

A common mistake in an AI-enabled website is putting the key into code sent to visitors. A person can copy an exposed key and use it in other tools, charging requests to the owner.

Bailey demonstrates with a pirate translator that sends entered text to a model. The sample deliberately uses an invalid key, so the request fails rather than exposing a working credential.

Viewing the page source shows the HTML for the input box and button, followed by the code that makes the model request. In the demonstration, a constant in that public source contains the fake key.

The request supplies the key, model and prompt asking for a pirate-style translation. If the exposed string were real, any visitor who copied it could use the account elsewhere.

07

Data centers became a local political issue

Bailey describes concerns about land, water and noise around proposed facilities. She cites 2026 ballot measures and polling to illustrate public opposition.

Bailey turns to the community effects of AI infrastructure, citing the November 2026 elections as an example of how data centers have become a political issue.

She describes opposition to a proposed large data center north of Salt Lake City involving investor Kevin O’Leary. Residents raised concerns about land use and water in a region already facing water shortages and pressure on the Great Salt Lake.

Cooling infrastructure uses water and electricity, and generators can create noise. Bailey says those local effects help explain why nearby residents may object to a project.

Bailey cites 41 data-center-related ballot measures across ten states on November 3, or 50 across thirteen states when earlier and later 2026 votes are included. She says 40 of the 41 November measures were decided against data-center proposals.

She cites an exit poll in which half of voters called AI data centers a crucial or major factor in their decision, and another in which 76% opposed a data center in their own community. She also points to spending on television ads mentioning data centers.

Bailey says data centers were a central issue in the Texas governor’s race, where the incumbent had welcomed construction as economic development. She presents the contest as evidence that local infrastructure impacts can shape politics.

She closes with Elon Musk’s post that data centers in space looked appealing after the election, relating it to opposition to facilities on Earth. Model choice and token use, she argues, should be considered alongside the infrastructure they require.

Sur la même chaîne

6 diapositives1:13:23EN
Stanford Online· 18 nov. 2026Terminal-Bench: How to Test Whether AI Agents Can Do Real WorkAlex Shaw, Ryan Marten · Laude InstituteThe benchmark's creators explain why useful agent tests need realistic tasks, reliable graders, and safeguards against shortcuts.ÉvaluationsBenchmarksIngénierie d’agents+2
6 diapositives58:52EN
Stanford Online· 17 nov. 2026Cynthia Bailey: Define What an AI Agent Can Do and When It Must StopCynthia Bailey · Stanford UniversityConnectors let models act through other apps; explicit permissions, human checks and stopping conditions determine how safely they do it.AI agentsMCPIngénierie d’agents+2
4 diapositives1:20:34EN
Stanford Online· 17 nov. 2026When to Optimize an AI Agent’s Prompt or Its WeightsMichael Ryan · Stanford UniversityMichael Ryan compares prompt search, reinforcement learning and distillation, starting with the evaluation that makes any improvement measurable.Prompt optimizationIngénierie d’agentsReinforcement Learning+2
1:09:27EN
Stanford Online· 17 nov. 2026Cynthia Bailey: Keep Testing AI After You ShipCynthia Bailey · Stanford UniversityOld test data can miss changed user behavior. Bailey shows how independent review, post-launch checks and better data expose AI risks.Agents de codeAI SafetySécurité et gouvernance+1
11 diapositives1:18:55EN
Stanford Online· 16 nov. 2026Michael Ryan: More AI agents can make a task harderMichael Ryan · Stanford UniversityParallel agents can explore more, but coordination costs and conflicting work can make a single agent more effective.Multi-agent systemsAgent orchestrationAgent memory+2
8 diapositives1:18:54EN
Stanford Online· 15 nov. 2026AI Agent Memory: What to Store, Retrieve and ForgetMichael Ryan · Stanford UniversityMichael Ryan compares agent memory architectures and shows why remembering more does not always improve decisions.Agent memoryAI agentsGénération augmentée par récupération+2