You Can Not Split The Atom In Your Head

Technology Anxiety, or The importance of starting before we feel ready, and seeing what fails

This is a post about how disruptive ideas are made real.

This is a post about AI Risk and the illusion of safety. It is the case for starting before we feel ready, because we only truly become ready by getting started. Building things allows us to experiment, and turn our abstract questions into real-world answers. We can only figure out what works by seeing what fails.

So how do ideas become objects? Perhaps you’d like to know the recipe.

First, we must start with a great idea, preferably a contrarian one. There are still plenty of blank spaces on the map, still unexplored because few have been crazy enough to venture there, and those who did were never heard of again. Pick an idea that used to be impossible, but now, through advances in understanding and technology, is merely extremely hard.

To our intriguing but virtually impossible idea, we need to add other people. No idea ever realised itself, it requires believers who’ll undertake the journey of discovery and will endure its terrible hardships. New things require craftsmen with the talent to manifest them, and leadership to guide, organise, and prioritise the collective effort. All involved must possess the wisdom to distinguish great from mediocre if they are to stay on the true path all the way to the end.

To fund all these individuals, we need not just a dream that’s inspiring enough, but sufficient cash to pay them. Building something that can scale up to change the world requires harnessing the potential energy of our economic system — what we call capital, the means of production. AI may be trained on data, but the substrate it runs on is more expensive than gold.

And then there’s one more vital ingredient to add, the courage to take risks. We’re not just talking about the opportunity cost of investing effort in something that might never work, but something unique to the most transformative ideas: what you’re building might be disruptive. It might even be dangerous.

Turning Ideas into Impossible Things

Soon after OpenAI’s May 2024 “Omni” event, as if to provide a coda to a momentous day, the company’s co-founder and chief scientist Ilya Sutskever announced his resignation. Ilya had mysteriously gone quiet after the attempted coup against Sam Altman late last year, and his whereabouts had become a bit of a meme for AI commentators. It was as if he’d seen something he wasn’t meant to see, becoming someone who knew too much, quietly “retired” to a Prisoner-esque village.

Sam Altman later posted a valedictory message praising Ilya as one of the greatest minds of our generation, saying that he left with his mission fulfilled. The mission in question was to build a multimodal model, capable of perceiving in any medium, and expressing itself in any medium too. But it also had to be more than a proof-of-concept, it had to be scaled up, beyond the point where something akin to reasoning emerges. The first version of that vision was what was launched as GPT-4o.

Sam credited Ilya with seeing the exceptional potential of LLMs early on. Ilya’s insight was that prediction about tokens was in essence a prediction about reality. He believed LLMs could process much more than language, and ultimately become the basis for human-equivalent intelligence. Once you make that intuitive leap, and believe it strongly enough, what once seemed like science-fiction became a series of engineering problems.

Geoff Hinton recently paid tribute to Ilya’s big idea: “Ilya was always preaching that you just make it bigger and it’ll work better. And I always thought that was a bit of a cop-out, that you’re going to have to have new ideas too. It turns out Ilya was basically right.”

In other words, now we know it’s possible to take the sum of human knowledge, process it with a simple algorithm, using prodigious numbers of exceptionally fast atomic scale processors, and generate answers without specifying how.

And talking of processors, nanoscale semiconductor fabrication provides another great example of how turning near-impossible ideas into physical objects requires an extraordinary synthesis of science and engineering. This time the impossible idea is giving sand the ability to think, which turns out to be possible by inscribing bafflingly intricate pathways of electrical circuits on highly refined silicon.

To digress for a moment, I don’t think it’s properly appreciated just how close to witchcraft the extreme ultraviolet lithography (EUV) machines that create our tiniest and most efficient processors actually are. Professor Chris Miller provides a great description of how they work:

“A ball of tin 30 millionths of a metre in diameter falls at several hundred miles an hour through a vacuum. It is pulverised by two shots from one of the most powerful lasers ever deployed in a commercial device and explodes into a plasma measuring several times hotter than the surface of the sun — several hundred thousand degrees Fahrenheit. This plasma emits EUV light at exactly the right wavelength of 13.5 nanometers, which is then collected via a series of about a dozen mirrors, which themselves are the flattest mirrors humans have ever produced. The mirrors reflect the light at just the right angle so that it hits the silicon wafer and carves the circuits on the chips that make [modern technology] possible.”

Without EUV, there’s no economical way to create nanoscale chips, and thus no petaflop GPUs, no multi-billion parameter language models, and no real-time AI. The machines that etch the chips are the size of train carriages, and require several jumbo jet shipments to transport the component pieces for assembly on the customer’s site. They cost about $150m each, and only one company on the planet, ASML can build them. And even ASML has no idea how to actually create chips with the machine it sells, because designing the chips is a whole separate science. That’s how complex the whole process is.

AI is a blend of maths and metals. The ongoing AI revolution is happening now not just because ingenious minds conceived new ideas, but because engineers have been able to build the machines that make them possible. Nanoscale chip fabrication has something in common with splitting the atom, it can only take place in machines, not in our heads.

The departure of Ilya Sutsever reflects a truism in all revolutions: eventually idealism is sacrificed for pragmatism. Doing the really hard things requires orchestrated teams of individuals, the deployment of capital, and the alignment of countless vested interests. Revolutions may start with a single idea, but they can only ever be carried out by coalitions.

The same day Ilya quit, Jan Leike announced he’d resigned too. He had led the alignment team at OpenAI alongside Ilya, which had recently been hollowed out by several senior departures. Super-alignment, by the way, is the goal of making future AI models incorruptible, so they intrinsically act for the benefit of humanity.

But super-alignment was always a slippery objective, even just to define. We can’t even get our own elected representatives to agree on the policies that best benefit tiny geographically distinct portions of humanity. I wonder if the internal safety culture was getting in the way of innovation, and degenerating into abstract philosophical arguments that couldn’t be tested.

As AI models improve, and are deployed more widely, it becomes increasingly important they do no harm. But the notion of complete safety is a mirage. As Sam Altman explained in a recent podcast, safety is never absolute, we do accept some things — like air travel — as being extremely safe, yet not flawlessly safe, in the name of progress.

Perhaps this is why the movie Oppenheimer resonated with so many. It captured the anxiety felt in any era of rapid technological change, when shocking new ideas suddenly become real. AI critic Max Tegmark has called GPT-4 a ‘Fermi moment’, and the Manhattan Project can feel like a lesson from history for many reasons. Sometimes what goes on in a single town does go on to shape the world, be it Los Alamos or San Francisco. No wonder Richard Rhodes’ Pulitzer Prize-winning book The Making of the Atomic Bomb is said to be a favourite of the modern day American Prometheuses of Silicon Valley.

Is AI Safety really Security Theatre?

Elsewhere in California, a bill called SB 1047 is currently passing through the state senate which proclaims: “If not properly subject to human controls, future development in artificial intelligence may also have the potential to be used to create novel threats to public safety and security, including by enabling the creation and the proliferation of weapons of mass destruction, such as biological, chemical, and nuclear weapons.”

This kind of moral panic isn’t motivated by a need to solve a specific problem, no AI model is about to help anyone build an atomic bomb in their basement. This is gesture legislation, a desire to be seen to be doing something. The reality is that you can’t mail order fissile plutonium, or build next-gen AI infrastructure like the Stargate supercomputer without a spare 100 billion dollars. Hazardous or not, complex things don’t just appear when you think about them. We aren’t the Krell of Forbidden Planet. We can not split the atom in our heads.

In popular culture, dystopias are far more commonplace than utopias. At the heart of all stories is the resolution of some conflict, and dystopian futures, being archetypal existential battles for survival, make perfect settings. Utopian worlds might initially impress us with their gleaming gracious greatness, but they soon become boring — because if trouble ever does break out in paradise, it’s no biggie, it is still paradise after all.

This imbalance means that through the movies and stories we consume, we collectively witness many more dismal depictions of the future than inspiring ones. It’s as if our minds have been trained to imagine threatening outcomes rather than optimistic possibilities.

The brilliantly bleak anthology series Black Mirror has an extraordinary dystopian episode called Metalhead, in which a woman is relentlessly hunted by a robotic quadruped the size of a dog. It’s plausible enough to be scary, as robots of this kind already exist, and the geospatial intelligence required to operate autonomously in unfamiliar environments is an active field of robotics research.

This episode is pertinent to this essay because deadly robots don’t just spontaneously emerge when technology gets advanced enough. They needed to be engineered for killing, assembled on production lines, and perfected by people through an iterative process of testing. Turning nasty ideas into real world objects doesn’t just happen one day, it is still incredibly challenging work.

Something else struck me when watching that episode. In fact, it’s a theme that occurs throughout the series: advanced technology is at its most dangerous when it is monopolised, and wielded by the strong against the weak. Metalhead would have been a very different story if the protagonist was accompanied by her own robotic guard dog. There is a very strong argument for democratising access to disruptive technologies, rather than hoarding it.

I believe the real risk of AI is not the creation of killer robots, but regulatory capture. It’s the danger of the technology being owned exclusively by a few, rather than serving the many.

Compute thresholds — meaning models above a specified level of complexity are subject to statutory vetting — are already in place in the US and EU, creating a two-tier environment for AI companies. The Guardian quotes one founder saying: “It’s better to be marginally under the threshold and avoid the faff of working with a regulator than to be marginally over and create a lot of extra work”.

In other words, we’re already bureaucratically hobbling our scientific and engineering progress. But why? What threat are we actually trying to mitigate against? No one actually knows, so AI regulation becomes just another kind of performative busywork.

In the recent Seoul AI Safety Summit, Andrew Ng argued that regulating AI made as much sense as regulating “electric motors”: “It’s very difficult to say, ‘How do we make an electric motor safe?’ without just building very very small electric motors.” He was not being facetious, the motors in fast-moving electric vehicles pose a far greater risk to life than any AI will in the foreseeable future.

AI regulation reminds me of the Red Flag Act, which was passed in the UK in the 1860s as motorised vehicles began to appear. It mandated that someone walk in front of any powered vehicle with a red flag, to warn those nearby of its presence, which meant the vehicles needed to travel slower than walking pace too. It remained on the statute book for 30 years.

So I am not a fan of AI regulation. It is security theatre, rather than a set of genuine safeguards. Knee-jerk AI legislation is counterproductive, it will reduce competition and make it more likely a few exceptionally powerful players emerge. Laws advantage those able to hire lawyers, and is that really our intention?

Poorly thought-out regulations simply create barriers for academics, early-stage startups, and open-source developers, whilst the richest companies will simply evade them, just as they do with their tax liabilities now.

Who’s Afraid of the Big Bad Database?

In a 2023 poll of 2000 UK adults, 29% said they feared “an advanced AI trying to take over or destroy human civilisation”, whilst 20% thought it was “a real risk that it could cause a breakdown in human civilisation in the next fifty years”. In another poll of US voters, 86% believed “AI could accidentally cause a catastrophic event”.

Anyone who’s actually built anything with AI recently will find this level of apprehension pretty weird. Modern AI is nowhere near as hazardous as these numbers suggest, and the notion we’re on the verge of super-intelligent machines emerging and running amok is an absurd fantasy.

So let’s take a moment to clarify what the term AI actually means. At time of writing, the foundational technology of modern AI is the Large Language Model (LLM), this is like a gigantic database containing most of what humanity knows, combined with an exceptionally clever query engine that allows users to ask questions in natural language rather than having to write computer code.

Thanks to the open-source project Ollama, I have several LLMs on the SSD of my MacBook. If you’d told me 10 years ago I’d have the sum of human knowledge stored on my laptop, I’d never have believed it. But it turns out that LLMs are exceptionally efficient means of compression.

What I certainly don’t have on my laptop is a malign artificial brain, operating according to its own unknowable agenda. LLMs are simply a novel kind of database. Are we really worried about really big databases taking over the world?

Yann LeCun, Meta’s chief AI scientist recently gave a pragmatic assessment of the capabilities of LLMs, which he said, had “very limited understanding of logic . . . do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term and cannot plan . . . hierarchically.”

That might make LLMs sound completely overhyped, but one of the great surprises of LLMs is how new capabilities — like reasoning — have emerged once models reach new magnitudes of scale, without being explicitly designed. But this reasoning ability is not synthesised from first principles, but adapted from the accumulated knowledge — which will inevitably include how-to guides and plans, which the model has encountered when it was trained.

Those worried about AI don’t seem to properly appreciate the massive difference in expertise between being able to follow a recipe, and being able to invent one that’s completely original. Remember that despite having access to the sum of human knowledge — literally every discovery ever published — LLMs are yet to produce a single original discovery of their own.

So whilst LLMs are brilliant inventions, they’re nowhere near the cognitive ability of the average human being, a rubicon known as artificial general intelligence (AGI). Many in the AI field, (LeCun prominent among them), believe new breakthroughs are necessary for AGI, that our brains aren’t based on the transformer architecture, so it’s unlikely any future AGI will be either.

Thus AGI is much more than merely an engineering problem, it’s still a scientific problem, as we lack fundamental understanding about how intelligence actually works. We’ll have to push back the veil of ignorance a lot further before we need to worry about AI becoming “threatening” and “replacing” us.

Perhaps a better term for AI is: intelligence-on-demand, a term that places human beings still firmly in control. Our intention is to be able to say to our machines, help me with this, or analyse and interpret that. We should be imagining a future where we delegate chores to our intelligent assistants to automate and optimise. That sounds like quite an attractive future to me.

That’s why AI regulation is such a frustrating folly. With anything new, you first need to build the thing. Only then can you work out how to make it safe, once you know what the real issues are — just as the pioneers of early aviation did in the first decades of flight. Building gigantic complex models allows us to evaluate new capabilities, and recent research is beginning to explain their fascinating behaviour.

The most transformative discoveries will occur at scale, beyond the compute thresholds that timid regulators and ignorant politicians want to police. We should not let irrational fears hobble vital scientific explorations. AI models are not powered by some arcane magic, their knowledge is simply an immense collection of 1s and 0s, and their answers are generated using the kind of maths teenagers learn. There is no reason to fear. There are no ghosts in our machines.

Is Technology Anxiety Really Capitalism Anxiety?

Writer Ted Chiang summed up our modern anxiety perfectly: “Most of our fears or anxieties about AI are best understood as fears or anxiety about how capitalism will use technology against us. And technology and capitalism have been so closely intertwined that it’s hard to distinguish the two.”

I am not an AI Doomer. I know there’s no ghost in the machine because I know how they work. Inside every LLM is a clever algorithmic process capable of exploring exceptionally complex mathematical spaces that have more dimensions than our minds will ever comprehend. But the final layer of a neural network outputs floating point numbers, not bullets or job termination notices. Built-in guardrails may prevent the most egregious abuses, but they can never guarantee whoever runs the model won’t use its results to terminate jobs and take away livelihoods.

That’s why AI Doomerism is really capitalist anxiety. When we worry about AI, what we’re actually worrying about is how their results might be used by a socio-economic system that quite often seems pretty inhumane. We’re not worrying that the AI itself is malicious, twisted by carbon envy and intent on crushing us. We’re worried because we know the system we inhabit already cares so little for us, and we’re anxious it doesn’t become even more algorithmic and compassionless.

It is inevitable AI will make many jobs economically obsolete, and that will cause social problems our societies will need to solve. Technological revolutions that destroy whole categories of professions are nothing new, but they happen because of the transformative possibilities they unlock. AI could give every child with a screen a world-class tutor, help us cure cancer or discover senolytic drugs, or develop green energy solutions to solve the climate crisis. We should build AI for the right reasons, not because we want cheaper stuff.

AI is still in its infancy, so it seems pointless to hobble systems that could turn into the most powerful tools we’ve ever wielded, just because their modern day incarnations might exacerbate the worst aspects of race-to-the-bottom capitalism, or offend people by generating hurty words. We as a species have long since learnt to be careful around fire.

Building revolutionary things requires the efforts of huge numbers of people, from technicians to experiment and quantify what works, to producers who generate the raw materials, to the engineers who build the factories and data centres. The idea must be manifested as material creations, by turning it into prototypes.

Some have proposed a CERN-style international non-profit institution to push forward the frontier of innovation for all mankind. Whilst nice in theory, there’s a reason AI is being driven by companies rather than institutions. AI advances fast through web-scale experiments, guided by the feedback of millions of users, and the financial rewards they suggest to investors. Companies are the very best means for rapidly iterating products, no bureaucratically controlled entity will ever outperform them.

Build Fast and Discover Things

It surprises me that many who must surely understand how AI works are still publicly calling for AI research to be paused, until we can be “sure” of its impact.

I disagree with this caution. I believe it ignores the reality of technological progress, that the implications of anything new only ever become apparent in retrospect. As all entrepreneurs know, once school ends and real life begins, the test always comes first, and the lessons afterwards.

Mistakes, failures and setbacks are how we learn what works. The unknown region has no signposts, only missteps and errors reveal the true way forward. We can not derive the implications of new things in a conference room, brainstorming beside a whiteboard.

The biggest risk of AI is, paradoxically, trying not to take any risks at all. That leads to a future when AI becomes a hegemony, fundamental to everyday life yet unchallengeable and controlled by the powerful — take a look at China if you want a sneak preview.

The safest future consists of millions of AI models, not just a tightly-controlled few. AI safety will not be guaranteed by government “supervision” and thousand-page laws, but by ensuring a diverse choice of intelligence-on-demand that’s accessible to everyone. There will be challenges, but the philosophy of the internet has always been to eschew centralised control, and when a problem occurs, to route around it.

The safest way to build AI is for everybody to build it, for the whole of humanity to contribute towards making it better — running live code in the real world, because we can not split the atom in our heads.

— Jaron, May 2024