The Strange Origin of AI's 'Reasoning' Abilities

In July 2020, a group of gamers on 4chan discovered that, when they asked a virtual character in AI Dungeon to solve a math problem and “explain it step by step,” the AI model not only arrived at the solution, but did so by assuming the role of the chosen character. The surprising detail: They were among the first people in the world to see in action what we now call “chain of thought” – the technique that enables large language models to explain the steps of a line of reasoning, not just provide the final answer. Today, big-tech marketing talks about “reasoning models” and chatbots that “think before responding” or are “able to reveal their thought process.” But the truth is that what appears to be a groundbreaking engineering achievement actually originated amid obscene jokes and memes on a notorious forum. The idea that AIs have learned to reason like humans is a much more recent narrative than the technical reality behind it. It all boils down to a misunderstanding: we think that the chain of thought is proof that a model is reasoning, because it gives us a detailed, step-by-step explanation. In reality, the model is merely imitating the texts it has read—including hundreds of thousands of solutions to math problems peppered with phrases like “Wait, no. The question is…”, “I should check the input first,” and “Wait, but in cases where…”. Rather than reasoning, it simulates reasoning. The protagonists of this story are not researchers from Google or OpenAI, but a group of anonymous gamers and a young computer enthusiast, Zach Robertson. On 4chan, amid over-the-top comments, someone writes: “It makes sense, since it's based on human language, that you have to talk to it like a person to get a sensible answer.” Meanwhile, Robertson published a post on how to “amplify GPT-3’s capabilities” by breaking problems down into multiple steps, and presented it in September 2020, unaware that he had contributed to one of the most celebrated breakthroughs in AI. Today, he is a PhD student at Stanford, but he seems to remember little about that discovery: his post had disappeared until someone pointed it out to him, and he is not interested in fame. Here's the crux of the matter: AI development companies have started marketing these models as “reasoning models,” but the real difference is not structural. Chain of thought works because it adds context: the more details there are in the question, the more the model is steered toward a precise answer. It's the same principle by which, if you ask ChatGPT a vague question, you often get vague answers. If you break the problem down into steps, the model has more clues as to where to go. In a study titled “The Illusion of Thought,” Apple demonstrated that these models can correctly solve a problem but fail if the question is rephrased with irrelevant details—in some cases, performance plummets by 65%. And sometimes, the chain of thought it produces has no real connection to the final solution. Some people say: If a machine fools us so well that it seems to be reasoning, then it must actually be reasoning. But the data tells a different story: the chain of thought is a linguistic trick, not a window into the machine's mind. If you thought chatbots “really thought,” this story forces you to look under the hood and see that they are often just playing a role. The chain of thought is a well-crafted act, not evidence of consciousness. On Lara Notes, there’s a gesture you won’t find anywhere else: I’m In. It's not a heart; it's not a thumbs-up. It's your declaration: this idea now concerns you. And if you discuss with someone how the AI thought chain originated among gamers and memes, on Lara Notes you can tag those who were there with Shared Offline—because certain topics deserve to be remembered. This is from The Atlantic: You just saved over three minutes compared to reading the original article.