Why Familiar Formats Can Improve Prompts

RÉSUMÉ

En 30 secondes

A speaker describes research that uses formats found in language-model training data to improve prompt accuracy. He offers a tentative explanation and an example of an unfamiliar separator.

1Use familiar formatsThe speaker cites prompt-mining work that found accuracy improvements from formats common in training data.
2A possible explanationHe speculates that familiar formats may make the task easier for the model.
3Avoid unusual separatorsAn unusual question-and-answer layout may be unfamiliar to the model.

À retenir

The speaker cites prompt-mining work that found accuracy improvements from formats common in training data.
He speculates that familiar formats may make the task easier for the model.
An unusual question-and-answer layout may be unfamiliar to the model.

ThèmesPrompt optimization

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

Mining common formats

The speaker cites prompt-mining work that found accuracy improvements from formats common in training data.

When choosing an exemplar format for a prompt, I suggest using a common format. I describe a prompt-mining paper in which researchers looked through a language model’s training corpus for common ways of asking and answering questions, then used those formats in prompts and improved accuracy.

A possible explanation

He speculates that familiar formats may make the task easier for the model. An unusual question-and-answer layout may be unfamiliar to the model.

I suspect the explanation may resemble the benefit of a smoother loss function during optimization: the space of questions may be smoother when the model has encountered the format before. That is my interpretation, not a proven mechanism in this clip.

For example, if I separate a question and answer with an unusual run of equal signs, the model probably has not seen that exact format often and may be more confused about what to do.