Can an AI check its own work? Two papers from Sakana AI, posted on October 8, 2026, test AI evaluating AI from different sides. One, called MASS, lets a model train itself using its own judgments. The other, called MLR, uses AI to review research papers. Both report strong scores in controlled tests. Both also show weaker results when the task gets closer to real use.
What MASS Does
MASS stands for Multi-Agent Self-Supervision. Researchers at Sakana AI and UC Berkeley used one open-weights model, Qwen3.6-27B, in three jobs. As a team of copies, it solves a task. It rewrites the team’s instructions. And it judges which result is better. The model is then trained on the work it judged best, and the loop runs again.
The training used synthetic research tasks in finance, robotics, and pharmacy, written by GPT-5.5. After two cycles, the model beat the base model on three held-out test tasks 69.9% of the time. Outside AI judges, GPT-5.5 and Claude Opus 4.8, decided the winners. On four public research benchmarks, score per output token reached 1.2 to 1.6 times the base model’s. The model’s own judge also agreed with the outside judges more often, rising from 73% to 93%.
Where MASS Is Weaker
On two coding benchmarks, Terminal-Bench 2.0 and SWE-bench Verified, score per token fell to 0.96 and 0.94 times the base. The authors call the transfer uneven. They suspect adding coding tasks to the training would fix it. That is a guess, not a result.
The headline efficiency claim is narrower than it sounds. A multi-agent student won 68.3% of comparisons against the base model using 28.8 million training tokens. A single-agent student won 64.0% using 42 million. That is 31% fewer tokens for a 4-point edge, pooled over five tasks.
The paper also warns about its own method. If a model grades itself, a shared blind spot could let a flawed result pass as an improvement and be trained in. If humans cannot spot the flaw either, later updates may reinforce it. The paper does not show this happening. In the parts I read, the outside judges were AI models too.
What MLR Does
MLR, short for Multi-Layered Review, is an AI paper reviewer. It runs on Claude models. One agent summarizes the appendix, an optional one searches the literature, and a review agent reads the paper in three passes. A review costs about $0.47.
To test it, the authors planted 1,164 contradictions in 257 published papers. GPT-4.1 wrote the errors, and the o3 model judged whether reviews caught them. On errors in a paper’s main claims, MLR caught 73.43% with four reviews. The best older system caught 14.81%.
Why 73% Is Not the Whole Story
Several details shrink that gap. The older systems used GPT models, and MLR used Claude. When the authors gave Claude to one older system, its score rose from 14.56% to 35.40%. So much of the gap comes from the model, not the design.
The 73.43% also counts an error as caught if any of four reviews finds it. One review caught 60.79%. Across all error levels, four reviews caught 40.95%. And planted errors may be easier than real ones. In a small audit, 34% of 50 sampled contradictions did not flow smoothly with nearby text.
On 211 real withdrawn arXiv papers, MLR matched the documented problem exactly 16.11% of the time, and 26.07% under a looser test. But this is not a clean comparison with 73%. The authors say withdrawal notes are often vague and the papers were mostly math and physics, which these systems were not built for. Still, detection clearly falls on real errors. All the reviewers were also swayed by hidden text in papers. MLR flagged it in 8 of 50 cases.
What Could Come Next
In the near term, others can check this work. The MASS code is public, so outside teams can try other models and tasks. The MLR authors say the tool should assist human reviewers, not replace them. In a user study with 38 sessions, authors agreed with 68% of its weakness comments.
The long-term picture is speculation. If AI cannot reliably judge itself on real work, self-improving systems will need checks from outside the loop. These papers do not settle that. They show where the controlled score and the real one start to part ways.
