An increasing number of AI companies are claiming that their models can reason. But two recent studies have reached opposite conclusions. When asked to demonstrate their logic, most models failed—demonstrating that they were less reasoning than repeating patterns. The answer turns out to be confident but not intelligent.
Apple researchers have discovered a key weakness in today's most popular artificial intelligence systems - they don't perform well when solving difficult problems that require step-by-step reasoning. In a new paper, the team tested several leading models on the ancient logic puzzle "Tower of Hanoi" and found that performance degraded as complexity increased.
The Tower of Hanoi puzzle is simple: move a stack of disks from one peg to another while following rules about order and disk size. For humans, this is a classic test of planning and recursive logic. For a language model trained to predict the next token, the challenge is how to apply fixed constraints across multiple steps without deviating from the goal.
Apple's researchers not only asked the models to solve puzzles, but also asked them to explain the steps to solve them. While most models only deal with two or three disks, their logic starts to break down as the number of disks increases. Models will misstate rules, contradict previous steps, or confidently make invalid moves—even when prompted by a train of thought. In short, they are not reasoning but guessing.
This finding echoes a study from April this year. At the time, researchers at ETH Zurich and INSAIT tested top artificial intelligence models to solve problems in the 2025 U.S. Mathematical Olympiad, a competition that requires fully written proofs. In nearly 200 attempts, no model could give a perfect solution. Among them, Google Gemini 2.5 Pro, which performed better, received 24% of the total score - not by solving 24% of the questions, but by gaining partial points for each attempt. OpenAI's o3-mini barely managed a score of 2%.
Not only do these models miss answers, they make fundamental mistakes, skip steps, and even contradict themselves while being confident. In one problem, a model performed well initially but excluded valid cases without any explanation. Other models have constraints designed into quirks of training, such as always framing the final answer—even if it doesn't fit the context.
Gary Marcus, a longtime critic of artificial intelligence hype, called Apple's findings "devastating for large language models."
"It's embarrassing that large language models can't reliably solve the Hanoi problem," he wrote. "If you can't use a multibillion-dollar AI system to solve a problem that Herb Simon, one of the 'godfathers of AI,' solved with AI in 1957, and that AI students can solve in their first semester, then the likelihood of a model like Cloud or O3 achieving general AI becomes slim to none."
Even when explicit algorithms are given, model performance does not improve. Iman Mirzadeh, co-lead of the study, put it bluntly: "Their process was illogical and not intelligent."
It turns out that what appears to be reasoning is often just pattern matching—statistically smooth, but not logically based.
Not all experts are dismissive. Sean Goedecke, a software engineer who specializes in artificial intelligence systems, found the failure instructive.
"The model immediately decides that 'manually generating all these steps is impossible' because it would require tracking over a thousand steps. So it keeps looking for shortcuts and ultimately fails," he wrote in an analysis of Apple's research. "The key insight here is that beyond a certain complexity threshold, the model decides that there are too many inference steps and starts looking for clever shortcuts. So, beyond eight or nine disks, the skill being examined quietly shifts from 'Can the model reason about the Tower of Hanoi sequence?' to 'Can the model come up with a general Tower of Hanoi solution that skips reasoning about the sequence?'"
Goedecke believes that these findings are not proof that the model is hopeless in reasoning, but highlight how artificial intelligence systems can adapt their behavior under pressure—sometimes smartly, sometimes not. The failure lies not only in step-by-step reasoning but also in abandoning the task when the reasoning becomes too complex.
Tech companies often highlight analog reasoning as a breakthrough. Apple's paper confirms that even models fine-tuned for chain-of-thought reasoning often hit bottlenecks once the cognitive load increases — for example, when tracking the movement of more than six disks in the Tower of Hanoi game. The internal logic of these models breaks down, and some models are only partially successful by imitating rational explanations. Few models can consistently understand causal relationships or goal-directed behavior.
The findings from Apple and ETH Zurich stand in stark contrast to the way companies market these models—as powerful reasoners capable of handling complex, multi-step tasks. In reality, so-called inference is often just advanced autocomplete with extra steps. The illusion of intelligence stems from fluency and formatting rather than true insight.
Apple's paper doesn't propose a comprehensive solution. However, it echoes the growing call for hybrid approaches that combine large language models with symbolic logic, validators, or task-specific constraints. These methods may not make AI truly intelligent, but they can help prevent incorrect answers from being accepted as fact.
Until these advances are realized, simulation reasoning is likely to remain exactly what the name implies: simulation. It's useful—even impressive at times—but far from truly intelligent.