ARC-AGI-3: Testing How AI Agents Learn the Rules of Unfamiliar Games

ARC-AGI-3 introduces an interactive benchmark designed to test how AI agents adapt to unfamiliar problems. In March 2026 its organizers reported that humans could solve the tested environments while early frontier systems scored below 1% under their evaluation setup. That result is a meaningful gap on this benchmark, not proof that every person succeeds or a complete measurement of general intelligence.

The ARC series aims to measure a particular kind of generalization: learning a new rule from a small amount of information and applying it to a new situation. I find this different from tests that mainly ask for an answer to a familiar question. Other challenging benchmarks also remain unsaturated; ARC is not the only one.

In an ARC-AGI-1 example, the test-taker sees a few input-output grid pairs. The examples show groups of three pink cells becoming complete two-by-two squares when a yellow cell fills each gap. Applying that inferred rule to the new input produces the proposed solution.

ARC-AGI-2 keeps the grid format but presents more difficult symbolic relationships. In the example I inspect, a colored key appears to encode the number of enclosed gaps in each shape. I reason through the examples and use that interpretation to predict the missing output.

I look at a leaderboard plotting score against cost per task. In the displayed March 2026 snapshot, strong systems achieve high scores on ARC-AGI-1 and lower scores on the harder second benchmark, with cost varying by configuration. These figures change over time and depend on the evaluation set and system; they are not current rankings.

Cost matters because the benchmark is interested in efficient skill acquisition, not simply getting an answer after unlimited computation. The organizers calibrate tasks with human participants, but a statement that each task or environment is human-solvable is not a claim that every individual solves every task or solves it effortlessly.

ARC-AGI-3 moves from static examples to interactive, turn-based environments. A player or agent receives observations and possible actions but no explicit description of the objective or mechanics. The challenge is to explore, infer what matters and use feedback to plan under action limits.

I start one game and examine the screen before moving. I see a yellow bar, a plus sign, a colored object and a symbol that may indicate a goal. I do not know yet which object I control or whether the bar represents health or a move budget, so I test an action.

Pressing up moves the colored object and reduces the yellow bar, suggesting that it tracks available moves. Going straight to the upper area does not finish the level. I notice the two symbols have different orientations and form a new hypothesis: perhaps touching the plus will change the orientation.

I reset, move toward the plus and see the goal symbol change. Once the orientations match, I continue to the destination and complete the displayed opening level. This demonstration illustrates exploration and inference on one level; the screen shows the game has additional levels.

I compare that experience with the displayed replay results. The selected game’s table shows a human baseline at 100% and several model runs at 0%; I then watch a GPT-5.4 run repeat moves without finding the demonstrated route. Those are specific runs in a particular harness, not evidence that the models can never solve any related problem.

The historical overall leaderboard I show has a leading result around 0.3%, with a substantial reported cost. ARC-AGI-3 scoring reflects level progress and action efficiency relative to human baselines rather than just the share of games completed. I mention the prize and invite viewers to try the different games; neither one level nor one benchmark settles the broader AGI question.