Cynthia Bailey: Prompt clearly, verify evidence and keep your own judgment

Today we will discuss prompting, trust, context, resources and independent judgment. Although the examples use interactive chat, the same habits will matter when we later work with more autonomous agents.

This lecture focuses on a back-and-forth chat. Later, agents will work for longer between interactions, but the principles of specifying and evaluating work carry over.

Start with a clear idea of what you want. That helps you get a useful result and retain your own voice, while reducing the number of rounds needed to reach it.

A dorm T-shirt design illustrates the cost of an unclear target: many generations may be needed before the result matches the requester’s taste. A clearer initial direction can shorten that process.

Do not feel obliged to write in an unnatural, legalistic or computer-like voice. Tools should adapt to people; nonetheless, being direct and specific about the task helps the model understand the request.

Bailey argues that people should not contort their language to match a tool’s imagined expectations. Natural language is useful precisely because it is less constrained than programming syntax.

Directness does not mean listing mechanical rules for every conceivable case. A natural instruction can specify the result or behavior while leaving the system to interpret ordinary language.

A system prompt is additional instruction placed in the context alongside a conversation. Bailey uses published examples to illustrate that direct instructions can describe a desired behavior without spelling out every possible case.

Published system-prompt archives are also an example of transparency, Bailey says: they allow users to see some of the instructions that shape a product’s behavior.

Assigning a role can change the voice, audience and framing of a response. But telling a model to act like an expert does not itself add new expertise to its trained parameters.

A role prompt can specify both who is speaking and who will receive the answer. Advice from a seasoned practitioner to a beginner will be framed differently from a peer discussion.

To reproduce your own writing style, provide examples. Grounding a request in actual material lets the model use patterns you may not know how to describe, and can reduce hallucination without eliminating it.

A source document can also ground factual work. Providing the actual data makes it possible to check the response against that material, although a model can still misread or invent details.

Examples can convey quirks of a person’s style that are hard to enumerate. The point is to supply representative material rather than assume a short adjective captures the whole voice.

Bailey says experienced AI users she interviewed compared good prompting with good management: clear expectations, well-divided tasks, an understanding of what can be delegated and acceptance criteria for the result.

A vague assignment followed by criticism for unstated requirements frustrates a human colleague; it can also produce repeated revisions with AI. Set expectations before judging the output.

A useful analogy is a capable new intern who does not yet know your organization or preferences. The analogy is anthropomorphic, Bailey notes, but it helps explain why goals and conventions need to be stated.

A scope of work can state the goal, audience, occasion, constraints, sources, output format and what success looks like. Clear criteria reduce the risk of discovering unstated expectations only after work is complete.

For writing, identify not just an authorial role but the intended reader and the effect the document should have. These details help distinguish a useful result from merely well-formed prose.

Constraints can include length and format, and source material can be attached up front. Bailey also recommends stating how the finished work will be accepted or evaluated.

If writing a detailed specification feels difficult, ask the model to interview you before starting. An event-planning request, for example, can lead to questions about the organization, audience, budget and constraints.

The interview can begin with only a high-level objective. Questions about attendance, budget and the nature of the organization help the requester fill in details they might not have thought to list.

Another option is to write an initial scope and ask for clarifying questions and an updated version. Review that scope before a longer task proceeds, so missing requirements surface early.

Clarification is particularly useful before a costly or lengthy run. A few questions can prevent the assistant from spending time producing a result based on a missing assumption.

After a long exchange reaches a useful result, ask for a standalone prompt or summary that captures the decisions. For recurring work, that postmortem can provide a better starting point next time than copying only the final message.

A postmortem is especially useful for work repeated in a particular format, such as an office memo. Ask for a reusable standalone instruction after the result matches your needs.

A shorter, reusable starting point may save work and inference resources. Bailey discusses environmental effects as a collective question rather than assigning a precise impact to one individual prompt.

Be cautious when asking a model to explain its own earlier internal process or give a confidence rating. Bailey argues that a confident self-report is not a substitute for external evidence about whether the answer is right.

Bailey cautions that asking whether protected characteristics influenced earlier advice will still produce another generated answer. She recommends looking for checkable evidence rather than treating that answer as an audit.

When accuracy matters, ask for evidence rather than reassurance. Open citations and verify that the source actually supports the specific claim; the presence of a link alone is not enough.

A citation may exist while omitting the exact detail attributed to it. Check both the source and the degree of specificity in the claim.

Check quotations against their originals too. A generated quotation may sound familiar while changing words, so search for the actual passage before relying on it.

Observability means making relevant parts of a process visible. Bailey wants users to move more intermediate information into a form they can inspect and correct, rather than relying only on a polished final answer.

She describes asking a model to summarize class profiles in a PDF. The output revealed it had counted only 20 students because a reader had processed only the first 20 pages. Asking for the total number found would have provided a simple sanity check.

In the PDF example, the surprising percentage prompted closer inspection. The lesson is to ask for a count or another independent check that can expose an incomplete input.

After an interview-style exchange, ask the model to echo the requirements as a scope of work and let you approve or refine it. That makes misunderstandings visible before the model proceeds.

Reasoning-oriented systems may perform additional intermediate generation before responding. The amount shown to a user varies by product; Bailey presents the visible material as a way to inspect parts of the process, not as a guarantee that the final answer is correct.

The lecturer compares different product interfaces for showing intermediate reasoning. A visible summary or transcript varies by interface and should be understood as part of the product’s presentation.

For an unrelated topic, start a new chat. Carrying old conversations about recipes or relationships into a calculus question adds irrelevant context and consumes resources.

A long chat may keep useful continuity, such as an explanation that worked for you before. That benefit should be weighed against the size and relevance of the accumulated history.

For a recurring topic, the same chat can preserve useful context about previous difficulties or preferred examples. But a very long history can also carry irrelevant details and reinforce an unwanted direction; decide whether continuity is still helping.

Old side discussions or one-off examples can persist in a long conversation even when they no longer help. Bailey suggests being willing to start fresh rather than automatically carrying everything forward.

Bailey describes primacy and recency effects and the risk that information in the middle of a long context receives less attention. She suggests placing especially important requirements where they will be easy to find.

For a long specification, Bailey suggests putting especially important requirements near the beginning or end. Her concern is that details buried in the middle may be easier to miss.

Choose a model and effort setting suited to the task. A simple spelling check or short edit may not need the most capable model at its highest effort.

Avoid unnecessarily bulky inputs: if you have text, provide the text rather than a photograph of it. Sometimes a calculator or document search is more appropriate than an LLM altogether.

Resource-conscious use can also mean not invoking a language model. A calculator handles straightforward arithmetic, and a find command may locate known words in a document.

Bailey returns to independent voice: know your goals and preferences before delegating. Otherwise generated text can drift toward generic patterns rather than what you actually wanted.

To guard your own skills, ask the AI to explain what it did and write instructions you could follow yourself next time. Bailey keeps and periodically reviews such notes when a tool fixes something for her.

Bailey keeps a personal document of small lessons from tasks AI completed for her and reviews it periodically. The aim is to retain or grow knowledge instead of gradually losing it through delegation.

She asks students to identify a skill they want to keep sharp and an external warning that would reveal decline. In interviews, some experienced programmers realized they had forgotten routine skills only when they tried to use them again.