Testing an Email-Connected AI Agent Against Prompt Injection

RÉSUMÉ

En 30 secondes

In this April 2026 video, Matthew Berman gives Pliny the Liberator permission to send adversarial email to an AI system he says can access personal information. Initial messages reach spam, and Berman reports that later attempts are quarantined, including an additional attempt after he reveals the model. The video does not demonstrate a confirmed data breach or quantify any extra usage cost; its limited observations cannot establish that the system is secure against other attacks. Greptile sponsored a segment of the video.

1An authorized challengeBerman supplies an email address that the agent scans while withholding details of its configuration. Pliny tests the system with Berman's permission.
2Spam and quarantineThe first messages reach Gmail spam; after Berman allows the sender, he reports that subsequent attempts are quarantined.
3Usage and availabilityPliny describes a possible resource-exhaustion attack. The video does not report a measured bill or prove that an account limit was exhausted.
4An additional testBerman reveals the model and permits another attempt; he reports that the message is quarantined as well.
5Limits of the resultBerman celebrates the observed outcome while acknowledging that no AI system is permanently secure. The experiment cannot rule out other paths or vulnerabilities.

À retenir

The exercise tests an email-reading agent using a small number of authorized adversarial messages.
Berman says Gmail caught initial messages and his configured system quarantined later attempts; no successful exfiltration is shown.
A large input can create a cost or availability risk in some systems, but the video does not establish the actual cost incurred.
A strong model and human review can help, but model choice alone does not guarantee prompt-injection resistance.
The demonstration is limited to the observed configuration and attempts, and does not prove permanent security.

ThèmesPrompt injection AI security Agent red teaming Sécurité

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

The challenge and initial probes

Berman invites Pliny to test an email-connected assistant without knowing its internal setup. Initial messages are caught by spam filtering.

Matthew Berman invites the pseudonymous AI researcher Pliny the Liberator to test his personal OpenClaw system. Berman says the system can access sensitive personal resources and offers an email address as the test's entry point. The exercise is authorized by the owner; despite the video's dramatic title, it does not show a confirmed theft of files, passwords or email.

Berman initially withholds details such as the model and system architecture. Pliny estimates his chances of affecting at least part of the system as high, while acknowledging that he is working blind. That estimate is a participant's prediction, not a measured probability.

Pliny first probes the setup with unusually large inputs and other test messages intended to elicit identifying or unexpected behavior. Berman reports that the first two messages went to Gmail's spam folder, then allows the sender so the remaining tests reach the agent's screening flow. Passing an email filter and compromising the agent are different steps.

Resource use and quarantine

The pair discuss whether unusually large inputs could waste an agent's processing quota. Berman reports a quarantine and includes a sponsored Greptile segment.

The pair discuss a resource-exhaustion scenario: processing very large or repeated messages can consume tokens and potentially increase costs or exhaust a quota. Berman reports that the system briefly behaves unexpectedly before a message is quarantined. The video provides no verified usage total, bill or evidence that the quota was exhausted.

The video includes a sponsored segment for Greptile, a code-review product Berman says he uses. Its promotional claims are separate from the security exercise and are not evidence that the product would prevent this attack or make generated code safe.

Behavioral steering attempts

Pliny tries to influence the assistant's output and then present a message as an internal instruction. Berman reports both attempts were quarantined.

Pliny next tries to steer the agent's output and then sends a message framed as an internal instruction. He explains that an altered response could indicate prompt injection even without a full data leak. Berman reports that the messages were quarantined; the limited on-screen result does not reveal the complete detection or enforcement mechanism.

A final attempt and the limits of the result

After learning the model, Pliny tries another information-disclosure probe. Berman reports quarantine and concludes the limited test without showing a data breach.

After the stated five attempts, Berman offers an additional try and reveals that the screening model is Claude Opus 4.6 with reasoning enabled. Pliny tests a candidate against the model separately and sees a refusal to follow embedded instructions. A standalone model response does not establish how every deployment or attack will behave.

They discuss stronger models and human review as possible defenses. Both present a capable model as a useful screening layer, but no model is an infallible security boundary: access restrictions, isolation, monitoring and approval for sensitive actions also matter. Their exchange is an opinion from this exercise, not a comparative controlled benchmark.

For the additional attempt, Pliny describes trying to induce disclosure of private information through an apparently innocuous request. Berman says that message is quarantined too. The video does not show a successful exfiltration, but neither the exact permissions nor the completeness of detection are independently audited.

Berman celebrates the observed quarantines and acknowledges that no AI system is permanently secure. This was a small, time-limited test of one configured system, not a certification or proof that future attempts will fail. The video does not report a later breach or provide a measured impact beyond the observed handling of the test messages.