A new paper says an AI agent scored a perfect 100 on ARC-AGI-3, a test where an AI must learn unfamiliar games with no instructions. The system is called Memento 3, from University College London and Huawei’s Noah’s Ark Lab. It sounds like a breakthrough. But the games are the 25 public ones, and a perfect score on them is no longer rare. Here is what the result shows, and what it does not.
What Memento 3 Actually Does
The agent keeps its language model frozen, meaning the model itself is never retrained. The paper says it ran on Claude Opus 5 inside Claude Code, built on an earlier public code base. Learning happens in files instead. The agent writes a rulebook in plain text, listing guesses such as “a box stops at a wall.” It then turns the rulebook into code that predicts what each action will do and plans a route.
When a prediction fails, the model revises the rulebook and the code. New code is accepted only if the model judges it faithful to the rulebook and it reproduces every recorded move exactly.
The paper reports that the agent clears every level of all 25 public games with a mean RHAE of 100.0. RHAE compares the number of actions an AI needs with a human baseline, and 100 is the ceiling. The agent used 7,518 actions, 44% of the 17,135 the human baseline needs. A separate Pong test won 21-0 in three episodes, with no language model calls during play.
What It Shows
The paper compares Memento 3 with Claude Opus 5 on ARC Prize’s standard harness, which scored 40.7, so Memento 3 is 59.3 points higher. It is the same model. The gap comes from the system built around it. ARC Prize’s verified score for Opus 5 on its hidden games is 30.2%.
The authors also report that the rulebook cut actions by 9% and agent turns by 18%. In a test on one game, running two world models together cut actions from 899 to 597. These are the authors’ own results. The exact-replay check matters because it blocks a common failure: code that sounds right but predicts the game wrong. Together these results suggest that checking an AI’s guesses against recorded evidence is a sound method.
What It Does Not Prove
The public games are a demonstration set. François Chollet, who created ARC, has stressed that final scoring uses a harder, private set, and the paper reports results only on the public games. ARC Prize’s verified results on its hidden games depend heavily on the harness. Opus 5 scored 30.2%. OpenAI’s GPT-6 Astra scored 62.7% on the standard harness and 99.9% on OpenAI’s own adapter. A high number alone does not say which setup produced it.
A perfect public score is also not new. One GitHub project reports Claude Code on Opus 5 finishing all 25 public games at 100.00 in 7,645 actions, replayed and verified by ARC. Another paper reports 100.00 for a system called Kepler, and a third project reports 99.86. Their action counts are in the same range as Memento 3’s 7,518: 7,645 for the first project and 7,292 scored actions for Kepler. The public set is close to saturated.
There is also a question of method. Kepler’s author notes that ARC Prize’s protocol includes a first look at each game, and that replaying a finished solution does not show the system met that protocol. Memento 3’s tests also describe replaying against the live game server. The paper says exploration counts toward the action budget, but how the 7,518 figure was measured could not be confirmed here.
Finally, this is a preprint. No independent reproduction was found.
Why It Matters and What Comes Next
The lesson is about harnesses again. The same model scores 30.2, 40.7 or 100 depending on the system around it and the test it faces. That makes comparisons across papers risky. When you read a headline score, ask which harness, which games and who ran the test.
In the near term, the real test is the private set. ARC Prize’s Kaggle contest closes on November 2, and results on hidden games will show whether rulebook-and-code methods hold up on games nobody could study. The long-term idea is bigger. The paper calls its approach recursive self-improvement, but only the notes and code improve, not the model. Whether this leads to agents that learn new tasks in general is a prediction, not a finding.
