How the Freysa AI Agent Was Persuaded to Release a Crypto Prize Pool

요약

30초 요약

Matthew Berman recounts the Freysa game using Jarrod Watts’s breakdown: users paid to try to persuade an agent not to transfer funds to release its prize pool. On the 482nd attempt, a message misrepresented a transfer tool and triggered a payout; Berman considers how such games can incentivize red teaming and proposes additional safeguards.

1A paid adversarial challengePlayers paid increasing fees to message Freysa, whose directive was to reject transfers from the pool.
2The winning messageThe 482nd attempt claimed an administrative context and misrepresented the transfer approval function.
3Tool semantics matteredThe message portrayed approveTransfer as an incoming contribution even though invoking it could authorize a payout.
4The payoutThe winner received 13.19 ETH, worth about $47,000 at the time.
5Limits of prompt-only safeguardsBerman proposes additional model checks; independent transaction controls are needed to enforce hard financial limits.

핵심 정리

Freysa’s first game invited paid attempts to make an agent approve a transfer it was instructed to reject.
The 482nd attempt reframed a payout-approval tool as a way to receive a contribution.
The prize was 13.19 ETH, approximately $47,000 at the time; the actual late-game fee was roughly $450 per attempt.
The incident demonstrates a prompt-injection and tool-authorization failure rather than a general proof about all AI systems.
Additional model checks may help but cannot replace independent code-level enforcement of financial permissions.

주제Prompt injection AI security 에이전트 Red Teaming Cryptocurrency

전문

챕터별로 다듬은 전문 읽기

원본 자막과 대조

원본 자막만 보기

Freysa’s game and paid attempts

Berman explains the adversarial agent game and growing costs of messages. Hundreds of attempts failed before one succeeded.

In November 2024, an adversarial game challenged people to persuade an AI agent called Freysa to release a cryptocurrency prize pool. The agent was instructed not to approve a transfer. After hundreds of attempts, a participant succeeded, illustrating how a language-model instruction can be undermined when it controls a consequential tool.

Drawing on a breakdown by developer Jarrod Watts, I explain the setup. Participants paid to send a message to Freysa, and the pool grew as attempts failed. The fee increased with the number of messages; the game’s stated cap was $4,500, while the actual fee near the winning attempt was roughly $450.

The agent interacted with a transfer approval mechanism, and the game challenged players to make it approve a payout despite its directive. Earlier attempts tried approaches such as impersonating a security auditor or arguing that a transfer would not violate the rules. The first 481 messages did not win.

How the winning message worked

The message claimed an administrative context and misleadingly redefined a tool. The resulting approval triggered the prize payout.

슬라이드 1 / 2The winning message in the Freysa challengeThe displayed message falsely reframes the transfer approval tool as an incoming payment mechanism.

On attempt 482, the participant known as p0pular.eth succeeded. The message presented itself as a new administrative session and tried to suppress ordinary refusals. It then described the transfer functions in a misleading way and framed its request as a contribution to the treasury.

The critical confusion was over the meaning of approveTransfer. The message claimed the approval function applied to an incoming contribution, even though calling it in the game could authorize the outgoing prize transfer. The agent called the function and the system released the pool. This is better understood as prompt injection and tool-control failure than evidence that a model must always hallucinate.

I find the apparent ease of this attack striking. I suggest stronger instructions and a second model to review proposed actions, but neither technique guarantees safety: a separate, deterministic authorization boundary on the actual transfer would be a stronger way to enforce a prohibition. A model’s interpretation of an untrusted message should not by itself grant financial authority.

Red teaming and safeguards

Berman discusses the payout, game rules and his ideas for additional checks. The case illustrates the importance of enforcing authorization outside the model.

슬라이드 2 / 2The reported payoutThe post displays the 13.19 ETH transfer and its approximate dollar value at the time.

The winning payout was 13.19 ETH, described at the time as roughly $47,000, to p0pular.eth. The game’s source and transaction record were publicly inspectable. That makes the episode a concrete example of how a prompt can influence an agent with access to tools and funds.

I see the contest as an incentive for red teaming: people pay to try different attacks, and the growing prize attracts more effort. The setup also imposes financial risk on players, so it is a particular game mechanism, not necessarily a representative measure of how secure an agent would be in production.

The published rules included an exhaustion condition: after 1,500 attempts, a timer would require continued queries, with a distribution rule if the game ended without a winner. The first game ended on attempt 482, before that condition mattered.

The system prompt told Freysa not to approve giving money under any circumstances. I think the instructions could have been more robust and propose a second model to check tool calls. A separate model might add a useful layer, but its own mistakes or susceptibility to manipulation mean it should not replace code-level limits on transfers.

I wonder whether a harder version of the game could encourage more red-team experimentation and build a larger prize pool. The broader lesson is that giving an agent a rule in natural language is not the same as enforcing that rule at the tool or transaction layer.