Containment of a laboratory accident

Imagine that ChatGPT's big breakthrough in November 2022 wasn't actually planned at all. OpenAI had only intended to conduct a test run for version 3.5, not to launch an official product. What really surprised the developers was that the system became so popular so quickly that, all of a sudden, all the major tech companies threw their reservations out the window. Up until then, caution, ethics, and technological maturity had been considered fundamental prerequisites. But after this “lab accident,” as it is called in the text, the only thing that mattered was securing the next big business model – even if that meant rushing the technology to market as a mass product. The billions invested in AI urgently demanded a “blockbuster,” no matter the cost. Even the idea that AI could soon cure cancer or solve climate change became a selling point – and if that wasn't enough, then it was adult entertainment. What is truly astonishing is that, after seventy years of AI research, we are now seeing language models that, for the first time, can communicate directly with us. In the past, AI was a chess master or an expert system for diagnoses. Today, you talk to a machine, and that feels both fascinating and unsettling at the same time. Suddenly, everyone—not just IT geeks—can talk to AI, and the expectation arises that this will finally make organizations more productive, more efficient, and more automated. But, and here's the catch: this new AI is anything but reliable. Businesses and government agencies are quickly realizing that they cannot rely on the answers provided by language models. Sometimes, instead of delivering precise answers, the machines just babble. And this is not because they function poorly – it is because they are designed to do exactly that: they produce plausible texts, but these texts are merely a combination of probability and chance. The bizarre thing about this is that AI is unreliable precisely when it is functioning perfectly from a technical standpoint. This is a real fallacy: We expect reliability from machines, but this AI doesn't deliver it. Of course, companies are used to dealing with uncertainties; employee mistakes or broken equipment are part of everyday life. But with traditional machines, there is always a logical cause that can be fixed. With language models, this certainty is missing—and that is a fundamental difference. Nevertheless, the myths about AI continue to be told: chatbots are supposed to free us from bureaucracy, automatically generate complete applications, and optimize processes. There is a great deal of euphoria, but people underestimate the number of precautions that need to be taken when using a technology that systematically makes mistakes and introduces uncertainty into important processes. Anyone with experience in businesses or government agencies knows that this relief comes at a high cost, because reliability, unlike with traditional machines, is not currently AI's strong suit. And yet, Silicon Valley likes to act as if the ultimate, perfect AI is just around the corner – if we would only invest a little more. What hardly anyone discusses is that it may be precisely the unreliability of AI that will become its biggest productivity killer. Or, conversely, that true progress lies not in perfecting machines, but in learning how to deal productively with their mistakes. Machines that work perfectly but are unreliable—that's what's truly new about this generation of AI. Today, when you trust an AI, you are actually trusting a black box that sounds plausible but can always be wrong. Once you understand this, you suddenly see the debate about AI in a completely different light. Machines that work but still get it wrong—that's the lab accident the entire tech world is now trying to contain. Machines that work perfectly but remain unreliable – that is the new reality of AI. If you recognize yourself in this, you can use I'm In on Lara Notes to show that this perspective is now part of who you are. And tomorrow, when you talk to someone about the surprising weaknesses of AI, mark the conversation on Lara Notes as Shared Offline – because it is precisely these real discussions that make the biggest difference. This text was originally published in the cultural magazine Merkur – and you've just saved yourself almost 10 minutes of reading time.