He Spent 20 Hours Crafting a Prompt. DSPy Beat Him Anyway

Nathan Labenz welcomes Sander Schulhoff back to The Cognitive Revolution. Sander founded LearnPrompting.org and organized the HackAPrompt contest the two discussed in January. This visit is about The Prompt Report, a 78-page survey of prompting techniques that have exploded alongside model capabilities. In the meantime the original tutorial site has become a startup selling generative AI courses to enterprises, and Sander is looking for a first full-time generalist to help with video and the business side.

Consulting-firm executives keep telling him the courses they get pitched are years out of date and miserable to repair. His workaround is to treat teaching videos like compiled software: write a script, have ElevenLabs read it, and when a customer wants a change months later, run one Python script that rebuilds the whole video. There is still no agent that can "make me a video about prompting, cover these topics, go to these websites" end to end. The team is building that pipeline themselves, with a lot of awkward tool-chaining in the middle.

HackAPrompt has not produced huge new attack families since January, Sander says. Microsoft's Skeleton Key looked a lot like techniques the contest had already covered. What did flare up is the argument over terms. He is "pretty set" on Simon Willison's original definition: prompt injection involves developer instructions in the prompt plus user input; jailbreaking is only the user talking to the model. The distinction is mostly about clarity and following the person who coined it, not about some high-stakes legal line.

He used to get the definition wrong. After HackAPrompt, Willison criticized the paper's wording; Sander reread it, agreed they were wrong, updated the paper, and wrote a Learn Prompting post about changing his mind. Nathan asks about Anthropic's many-shot jailbreak and about Pliny, who pulls system prompts out of consumer chat UIs. Sander is unimpressed by many-shot: HackAPrompt contestants already stuffed in piles of few-shot exemplars. Under Willison's definition, Pliny's work is mostly prompt injection, even though Pliny and the rest of the scene call it jailbreaking.

Nathan had hoped OpenAI's hierarchical instruction-following — system prompt first, then the user, in a cascade of priority — would lock this down. Pliny has argued the opposite: as models get more capable they get easier to inject, in part because there is more surface area, including prompts embedded in images. Sander has no pet defense. He liked a Lakera contest that used an adaptive system prompt, absorbing each successful injection the way a vaccine absorbs a pathogen, but still thinks "defense here is always a losing game."

He beta-tested Haize Labs after dismissing automated jailbreaking a year earlier, then watched the product jailbreak arbitrary intents — bomb-building instructions, racist output — in seconds. It was the first team he had seen actually do that. Training a second model to detect attacks does not impress him. The problem, he thinks, gets solved by model creators in training or architecture, not by app-layer filters. Nathan floats Anthropic's Golden Gate Claude-style feature detection and Dan Hendrycks's circuit-breaking work, which reroutes harmful internal representations toward refusal. Sander will not bet on a specific path.

The paper started as a pivot. Working under Professor Philip Resnik at the University of Maryland, Sander wanted to measure how much generative AI actually makes people faster at a task. He found about thirteen such studies already, with sample sizes from eleven people to a few thousand, then realized a proper experiment would take six to nine months — too long for the semester. A prompting survey, he told himself, could be done in three or four months. "How hard could that be?" It took nine.

He recruited from UMD's NLP lab, posted in the Discord for Startup Shell (a student-run incubator), brought in a collaborator from OpenAI, emailed paper authors as he read, and late in the project added four suicide-prevention specialists for a case study on detecting suicidal intent in social media posts. Advisers suggested topic modeling to see what the literature actually cared about. The topics lined up with the sections he had already sketched: text prompting, then multimodal, multilingual, agents, and evaluation.

Letting GPT-4 read all of arXiv and decide what counted as a prompting technique was too expensive. The team searched arXiv, Semantic Scholar, and the ACL Anthology with 44 keywords, deduped the hits, and had humans review about a thousand papers. Anything that required fine-tuning was out. They used those labels to prompt a model to screen the rest, then kept pulling new papers throughout writing. Co-author Michael Ilie led the pipeline. A full run took hours, so a sloppy test wasted both time and money.

Nathan control-F'd the PDF and confirmed that the word "delve" never appears. Sander did not search for it. Humans wrote the paper, though people likely used ChatGPT in small doses, and he never ran a find-and-replace on telltale model vocabulary. What he did care about was quality. On a large research team, he would hit passages where someone had clearly generated text and not fixed it: the wording did not make sense, and you could tell. That, not the use of AI itself, was the problem.

He did at least five passes over the 70-plus pages, including a final word-by-word read, to strip the bad generations. He is actually surprised there are zero delves left; he would have expected a couple. Then he walks through the taxonomy of advanced techniques, most of which need code or a pipeline: few-shot, chain-of-thought (a family of generation tricks, not one trick), decomposition, role prompting, style prompting, ensembling, and self-criticism. Whether to merge decomposition with chain-of-thought was a real fight, because both break a problem into steps. Another graph, with techniques living in several categories at once, would also have been valid. Nathan jokes that the field needs a sparse autoencoder for prompts, to see which techniques are "active" in any given string.

The diamond in the rough, for Sander, is few-shot design, plus a definitional cleanup the literature still needs. Few-shot prompting and in-context learning are not the same thing. Under Brown et al. 2020 — *Language Models are Few-Shot Learners*, the GPT-3 paper — asking ChatGPT to "tell me a story about a dinosaur" already counts as in-context learning, because specifying a new task in the prompt is the point. Models used to be trained for one job; now you describe the job and they do it. Few-shot is only one form of that. He has read the paper more than twenty times and had the OpenAI collaborator check internally. Other papers that cite Brown still mix the two up.

Figure 2.3 in the report lists six pieces of advice. Each can hurt accuracy on a given run; in aggregate they help. Quantity: more exemplars are better, balanced against cost and latency. Order: randomize. Put all the positives first and all the negatives last, and the model may keep predicting the class it just saw. The same set of exemplars can yield anywhere from 0% to 90% accuracy depending on order — "super surprising and honestly quite frustrating," because there is no reliable way to find the optimum. Nathan asks whether a carefully designed order can beat random. Sander says usually not, and those wins look like luck.

Label distribution should be balanced for a binary task, unless the real world is skewed. If 90% of cases are positive, a 9:1 exemplar mix can raise overall accuracy; match the data, do not invent a fair coin. Label quality still matters. Some papers find wrong labels do not hurt much, but people auto-build prompts from messy datasets, so clean labels are how you give yourself a chance. Format: pick a common one. Sander almost always uses `Q:` input, `A:` output. A prompt-mining paper found that matching question-answer formats frequent in pretraining data improved accuracy, which he analogizes to optimizing on a smoother loss surface. Exotic separators the model has never seen make that surface rougher.

Last is similarity. At inference time, retrieve exemplars similar to the test instance and drop them into the prompt — a RAG-style dynamic prompt. Other papers argue for diversity; Sander's experience is that similar exemplars are more reliable, with diversity helping only in specific cases. He also nits the vocabulary: an item sitting in a dataset is an example; once it is in the prompt it is an exemplar. The team's own experiments ran mainly on GPT-3.5, with some GPT-4. Running every technique on every model they wanted would have taken about a month on public APIs. He still thinks the six rules travel across OpenAI, Claude, and Google models, and that every technique in the paper can be implemented with a standard LLM.

Nathan asks how real the folklore is that an extra space between sentences can tank a prompt, even at temperature zero. He has been a two-spaces-between-sentences person for years and would reconsider for model performance. Sander, a one-space person, says it is a real reliability problem, discussed in the report's reliability and alignment section, but it should fade as models improve. GPT-4 is much more robust than GPT-3. When he writes prompts he does not ablate single words; he assumes small edits will not matter much, even though he knows they sometimes do.

He does not credit RLHF with hammering those oddities out. RLHF, he argues, has been shown to decrease accuracy — a tradeoff between being right and being pleasant to talk to. Nathan's experience matches: base models beat RLHF-tuned ones at joke writing, imitating a famous writer's style, and random-number generation. Whether coding gets better depends on the learning signal. If the reward is "does the code run," that is strong. If it is "does a human like how the code looks," the signal is weaker, and you should not expect a free accuracy gain.

Sander has been frustrated by GPT-4's refusals while trying to understand how credit scores are calculated. The model called the formula proprietary, then said even an example formula "might be harmful." He still mostly uses ChatGPT and Claude for coding, not base models, unless a research paper requires them. Nathan has been comparing Llama instruct versus base on Together AI and suspects base models win when you want a high-variance, outside-the-box idea — a punny podcast title, a weirder joke — rather than a high average score across all generations.

Nathan's default advice for a messy task is to sit down for a few hours and write gold-standard examples with painfully explicit reasoning: the cues, the logical jumps, then a second pass that is even more explicit. Most real work is not a multiple-choice benchmark with a ground-truth letter. Sander agrees almost verbatim. The most common failure he sees is people dumping instructions and hoping. "Showing multiple, at least three examples of what should be done can really, really help."

Ensembling mostly assumes a discrete right answer: run the model ten times and take the majority. Nathan wants a way to spend more compute on open-ended work, like Waymark video scripts for small businesses, where there is no single correct output but there are clearly better and worse ones. Sander's answer is frustrating. You can pipeline a draft, then a revision prompt with guidelines, then criticize sections separately. You can assign roles to multiple agents — writer, grammar checker, style checker. He has read multi-agent debate papers and does not think they are worth implementing yet. He probably would not implement the self-criticism pipeline either.

Nathan's alternative is to bootstrap into fine-tuning: generate with the best model, have humans filter what they like, fine-tune, then expand coverage of edge cases. At Waymark, the biggest lift was fine-tuning on chain-of-thought reasoning rather than input-output pairs. Stuffing twenty exemplars into every prompt is more expensive and slower than a fine-tune, even though the fine-tune itself is not cheap. Evaluation still is not automated. They have structured evals they do not fully trust, because of edge cases, and they still look at the data and use the product themselves. Sander likes the idea about as much as the prompting pipelines — it can help, without being a sure thing.

On multilingual prompting, the boring advice is still: translate into English, do the hard part, translate back. Sander's favorite trick from the chapter is multilingual ensembling — run the same task in several languages and combine the results. Nathan had never seen that before; it was Sander's favorite too. There were no other big surprises he wanted to flag from that section.

In multimodal work, his favorite is chain-of-image prompting. For a visual reasoning question — does the line *y = x* intersect this circle? — the model graphs both, renders an image (say via an SVG Python package), looks at the picture, and asks itself whether they cross. It is chain-of-thought with images as the steps. Video is another story. Runway's Gen-3, in his hands and in a friend's who specializes in video prompting, is "extraordinarily difficult": you have to be painfully specific, closer to early GPT-2 or GPT-3 prompting than to ChatGPT. He has not tried Sora or Kling.

Nathan notes the density: whole paragraphs of composition and transition notes for a few seconds of Gen-3 video, similar to how DALL-E 3 expands a naive user prompt behind the scenes. On image inputs to language models, he has had more luck overlaying grids, segmentation maps, or dedicated OCR. Frontier models reading scanned plumbing-parts catalogs mangled part numbers; OCR the numbers to text, keep the diagram, and the same models suddenly return the right SKUs. Sander has mostly stayed in text, aside from recent content-generation experiments.

Conventional wisdom, which Sander shares, is that agents do not quite work. He likes simple loops such as ReAct and thinks tool use can be reliable when the tools are very specific. What he wants next is architecture that makes tool use a first-class act, not a fine-tune that emits a magic string, plus reinforcement learning so models learn to act and generate their own reward signal. "Hopefully eventually RL will have its NLP/LLM moment."

In practice even he finds agents frustrating. He cannot say, "I just got an email, I need to pay someone, go to my bank website and send them this much money." That should be a few URLs strung together; it is not available. Nathan's one-liner: it also sounds like a fast way to drain your bank account. Sander agrees the security concerns are huge. He has no hidden gem from the agent chapter beyond that — lots of approaches, some narrow tasks that work somewhat well, and nothing that does the simple multi-site chore people actually want.

Late in the conversation Nathan unburies the lead. Sander spent about twenty hours building a prompt for a binary classification task and was happy with it. His adviser then ran DSPy on the same exemplar data. Sander is unsure how to pronounce the name — "dispy," or letter by letter; Omar Khattab spoke at Maryland and Sander still cannot remember the official pronunciation. DSPy produced a prompt, from the same data, that "blew me out of the water on the test set."

Automated prompt engineering works, he can now say, if you have ground-truth labels to optimize against. He does not think it can optimize just any prompt, because it needs a signal to climb. It is a black-box framework: it can call proprietary model APIs and does not need weights. He gave it fewer than twenty exemplars; the dataset itself was only a couple hundred items. "I definitely didn't think I could be defeated by an automated prompt engineering system." If you have labeled data, he thinks you should use it.

Nathan wonders about productizing a prompt coach that follows you around with the taxonomy. Sander is leaving that to others for now. Learn Prompting has a RAG chatbot over the docs, but scoring arbitrary domain prompts consistently — and getting people to pay for it — is unclear. His management takeaway from a 360 review is sharper: "Trust no one, not even yourself." In practice that means tests and a CI pipeline on every pull request, including his own, so style checks and regressions cannot sneak through. He would not recommend another systematic literature review; it took twice as long as planned. He is still glad he did it, and glad the community has the map. The Prompt Report is 78 pages and free.