Can a Language Model Keep Track of a Variable? What Ai2’s StateBench Reveals
The new Ai2 StateBench test shows top AI language models failing at complex memory tasks, dropping to 13% accuracy on long documents. The reason might be their core architecture.