AI Adventures in The Library of Babel

This essay is about Generative AI, the vastness of possibility spaces, and the search for an answer to the following riddle:

Would infinite books give us infinite knowledge?

Where better to find the solution, but within an infinite library? After all, it contains every book ever written. Every idea that will ever be thought will be contained within its shelves.

You can come with me if you like. Shall we go on an adventure?

Just paste the following into your favourite AI model, ChatGPT is good, it also works for Bard, and Bing.

Let’s play a text adventure game, set in Borges’ story The Library of Babel. At the start of each turn, you will describe my current location. You will then ask the question with the heading “what do you do?”, and then provide me with three numbered actions I can choose from. Begin by telling me my starting location.

Then, you should see something like this scribbled across the screen…

> 1

> 3

> 1

> 1

> 1

> 3

> 1

> 3

> Go back to the code

See how you can go off-piste, and improvise your own actions? The three options are only suggestions after all. This produces the following reply:

How ironic the options here are to enrich yourself, make the world a better place, or suppress the discovery. Because humanity is facing exactly the same choices when it comes to how we handle the revolutionary change brought by modern Artificial Intelligence.

What would you do?

The Possibilities are Endless

Now, an interlude, as I introduce some terminology and explanations that will be helpful for later.

The software that created the above responses isn’t a computer game, but a special kind of large language model (LLM) called an instruction-following Foundation Model. As their name suggests they’re designed to perform far more than just a single task, they can be given instructions in English (what’s known as a prompt), and be told to create literally anything you can imagine and can put into words. No coding necessary.

That’s why in 2023, the hottest new programming language is English.

As these models create new things, they’re collectively termed Generative AI, or Gen AI for short. Given this will be defining technology of our generation, I think Gen AI is a pretty apt name.

As the example above demonstrates, any story can be turned into an interactive adventure. I only needed one short prompt with two sentences. One to describe the setting — in this case, Jorge Luis Borges’ classic short story The Library of Babel, and one to specify that I expected to interact with the model as if it was a text adventure.

Were Borges alive today I like to think he’d be running his own esoteric Gen AI experiments with a wry smile, he always did love the notion of a garden of forking paths.

At time of writing, this only works for famous and influential stories that your AI model has already encountered during its training. If you supply a URL and ask it to read it, it’ll just make up (hallucinate) a story based on its title. This is a deliberate hobbling, public AI models can’t access the internet at runtime (for “safety” reasons).

Until recently, all Gen AI has been trained on general knowledge, such as the text of books, and publicly accessible web pages. A week before I wrote this, OpenAI announced support for plug-ins that would allow external sources of information to be incorporated, (like previously unread stories). This will be a major advance, allowing new highly specialised sources of information to be integrated, permitting Gen AI to incorporate knowledge it hasn’t been trained on. Models will be capable of outsourcing, they’ll become synthesis engines.

So that’s the current state of the art, as of March 2023. This may all seem rather quaint when I read it back in a few years’ time, but that’s why I wanted to write this down now, to document a moment in time, at the cusp of what seems likely to be a revolutionary change.

Now, let’s get back to talking about adventure games.

That Gen AI can improvise adventure stories has been well-known for years. The pioneering AI Dungeon was created using GPT2 in 2019, and updated to use GPT3 a year later. So being able to jump into and inhabit any story is a neat demo, but no great revelation. But this essay isn’t about the capability to improvise, it’s about the nature of possibility space and the riddle of infinite books.

Whilst my own short AI-generated journey through Borges’ library was diverting, one considerable omission was a lack of destination. It’s ironic that I really did feel like I was trapped in a library of infinite stories from which I could never escape — because that’s exactly what any endlessly improvised story ultimately turns into if you don’t supply an ending.

One aspect I found quite fascinating was how the model tried to inject some drama into the ongoing narrative. There are no toppling bookcases in Borges’ original story, but danger is a staple trope of choose-your-own-adventures stories, and what peril could you possibly encounter in a library? Does Bookcase + Peril == Toppling sound like a reasonable formulation to you? Good, because that’s pretty much how modern AI models work.

So, yes, these kinds of models do allow us to inhabit books, just as William Blake once predicted, we can now find whole worlds in a grain of sand. Or a single word, like, for instance, “Dickensian”.

Try this prompt in your favourite foundation model:

Let’s play a text adventure game, in a Dickensian setting. At the start of each turn, you will describe my current location. You will then ask the question with the heading “what do you do?”, and then provide me with three numbered actions I can choose from. Begin by telling me my starting location.

It’s the same prompt as before, only the setting has changed. This produces the following output:

This time, I left the alleyway and followed the smell of roasting chestnuts, leading me to strike up a conversation with a chestnut vendor. I bought some nuts to give to a hungry nearby street urchin, and you’ll never believe what he told me, and then… well, you get the idea.

Once again, we find ourselves wandering around a seemingly infinite library, except this time it’s a simulacrum of Dickensian London. Of course, if we really wanted to turn this into a proper adventure game, we’d need to add challenges and puzzles, which contribute towards achieving the final triumphant objective. And I think that is an important point when it comes to the creation of derivative works.

The world doesn’t need a million new autonomously generated variations of Dickens’ works, so if we want to generate something new from existing works, we should only do it if we also add something new of our own.

Why We Must Create New Things

Imitation has been a fundamental part of Art for millennia. Artists inspire subsequent generations, each new creator adding their own unique interpretation to well established forms, like The Portrait, or The Landscape, or The Crucifixion. But every now and then, some radical creates an entirely new genre, inspiring new generations to create works that have never been imagined before.

The key point here is artists might imitate, but they also build upon what’s gone before. If they didn’t, our culture wouldn’t evolve, there’d be a standard (but archaic) way to write, paintings would be resolutely figurative, and nobody would dare to go abstract. We’d have stopped creating anything genuinely new centuries ago.

Culture is humanity’s shared commons, and like all commons, relies on everyone to protect and preserve it. We have a duty to use our new Foundation Models responsibly, and the clue is in the name, they should provide the foundations on which we build new things, not become automated counterfeiting factories churning out knockoffs.

There’s nothing wrong with remixing, that process of culture sex for where we combine the genes of disparate ideas together, and see which of the progeny thrive and flourish. Artists who remix are like the searchers in the Library of Babel, trying out different combinations, and assessing whether they have soul.

Let’s consider this thought experiment. Suppose that instead of creating a derivative Dickens story, we used the power of Gen AI to integrate disparate historical sources, and thus create a highly detailed reimagined Victorian London?

Perhaps we’d start on the busy Whitechapel Road, drawing on information from Charles Booth’s maps of income inequality, years of local newspapers reports, and census records. Gen AI could make all of this up of course, but how much more compelling would it be to create a world from real sources of information? We’d be able to knock on any door, look through any window, talk to named characters who actually existed, and learn their hopes and fears.

All games need objectives, so how will you respond when you find yourself lodging with a kindly but haggard parent, and discover their child is desperately sick, and they can’t afford the treatment? Coldly walk away? Offer to earn some money to help pay for it? Help set up a trade union and demand higher pay and conditions? Burgle the rich and give to the poor?

If that example sounds a bit facetious, like some kind of Grand Theft Dickens, remember that the best-selling open-world game, GTA5, is itself a remix of 70s and 80s cultural tropes (particularly mob movies). Estimates say GT5 has earned $7.7 billion since its release, more than any movie ever made. What if we played in socially realistic worlds, rather than imaginary battlefields?

I believe Gen AI will be the engine of future games because it solves a fundamental limitation that exists in every game ever created. If you zoom in close enough, you can always see the pixels. These might be actual pixels, or metaphorical pixels, but sooner or later, if you try to do something that’s not been modelled or scripted, the illusion of immersion breaks.

Just as low resolution graphics are interpolated and upscaled to high definition, Gen AI can upscale scripts, characters, locations, and ideas. I think Midjourney founder David Holz put it well when he said he believes in the near future: “you’ll be able to buy a console with a giant AI chip and all the games will be dreams.”

Of course, we can’t just tell Gen AI to “write me GTA5, but with a Dickensian setting”. Modern games are highly complex pieces of software, precisely engineered to be fluid, challenging, and enjoyable. Designers will be needed to write the game dynamics, and artists will still be required to imbue the world with wit and a soul.

But bit by bit, the labour required to build these worlds will be automated. Amongst the first to go may be motion capture, 3D models, and raytracing. As Jensen Huang, the CEO of NVIDIA recently said: “Every single pixel will be generated soon. Not rendered: generated.”

There’s a lot of anxiety about Gen AI at the moment, a worry that it will destroy jobs and overwhelm us with credible sounding but incorrect noise. It is a risk, but the AI Genie is already out of the bottle. All we can do is take personal responsibility and command it ethically, in a way that does no harm. Because we could use it to create such great things.

Work is for Machines

I wanted to use the example of games in this essay because in a way, every piece of software we use is a game in disguise. In them, we wander through possibility space (everything that can be imagined) looking for opportunities to accumulate value to ourselves, or our company. It’s just that our everyday game board isn’t an awe-inspiring space station or a fantasy castle — it’s a spreadsheet, a dashboard, or a text editor. And often the activities we perform aren’t really that much fun at all.

Gen AI is coming for busywork, and good riddance to it. We’ve finally given machines the map of language understanding, and they can explore these possibility spaces far faster and so much more extensively than a human being ever could. As we’ve seen from our discussion of games, Gen AI is adept at world building, it can combine data from different sources, it can synthesise answers to our questions.

If we can delegate busywork, what does that leave us to do? No one is suggesting we’re all about to retire to lives of luxurious leisure. Work brings essential challenges to our lives, it provides the exercises that are vital to our continuing personal growth.

In Bertrand Russell’s classic essay In Praise of Idleness he wryly observes there’s really just two kinds of work, moving things about, and telling others to move things about. Instructing machines is the future of human work. We shall be the arbiters that decide whether the results of the machines’ effort are any good or not.

The utopian view is that every worker will become management. The dystopian view is people will be cheaper than software, and so every worker will receive their orders from automated management — like how food delivery apps work. We’d be wise not to discount this dystopian future.

Russell’s contemporary John Maynard Keynes thought we’d have a 15 hour working week by now. Alfred North Whitehead said “Civilization advances by extending the number of important operations which we can perform without thinking about them.” That should be the future of work, continuously asking ourselves: “Why am I actually doing this schlep?” and “How do I automate this task away?”

Once we delegate busywork to machines, what would that leave us to do? Borges suggests one answer in The Library of Babel. The searchers are curators of that universe, they spend their lives looking for what Christopher Alexander called the Quality Without a Name, that ineffable sense of something meaningful.

We will be like the explorers of the Library of Babel. Our jobs will be to find meaning within the limitless spiel AI generates. Someone must arbitrate between hallucination and truth, to direct the search towards what we really seek. To enforce ethical standards, and guard against exploitation. We will be overseers, assessing what is produced and deciding: Is this really helpful? Asking: Is this really any good?

Generating noise is easy. Writing a symphony that moves the soul — now that’s hard.

The Illusion of Infinite Content

The moral of The Library of Babel is that abundance is not always a blessing, and can actually be a curse. The vast majority of books in the library are worse than worthless, because their sheer number obfuscates the miniscule number of books that do actually contain valuable meaning. Only a tiny number of books contain authentic truths, and they are sunk deep within an ocean of gibberish and deceptions.

The parable warns us that for every profound truth in the library there will be countless other variations that are subtly wrong or treacherously misleading. Just because a book appears grammatically correct doesn’t make it a treasure, the majority of what glitters in the infinite library is actually Fool’s Gold.

Borges’ parable reminds us that art is so precious because works with soulful meaning are so scarce, just finding one might take a whole lifetime.

Human beings are the sole wellsprings of original thought. Even the words you’re currently reading might exist as a manifold somewhere inside GPT4’s incomprehensibly vast multi-dimensional space, just as it would inevitably also exist somewhere upon The Library of Babel’s infinite bookshelves. But you’d never be able to write a prompt to generate this exact sequence of words, in reality this essay required a specific human being to craft it. Me.

As Borges himself once wrote, when pondering the notion of being Kafkaesque: “If Kafka had never written a line, we would not perceive this quality; in other words, it would not exist.”

It might seem like we’ve created our own Library of Babel, as our new technology can now conjure into being an infinite number of books. Gen AI can generate stories in any setting, in any style, turning the seed of a few words into a whole forest of prose. But there’s a subtle difference: the Library of Babel consists of every book that will ever be written, but Gen AI can only create variations on the accessible subset of public content that exists right now.

This is a vital and greatly underappreciated point. The possibility space of what Gen AI can imagine is so vast it appears infinite — they seem to be able to do anything! But actually, its imaginable possibilities are distinctly finite, because there’s only so much high quality data to train them on.

A recent paper estimates the amount of high quality training data available is about 9 trillion words, give or take a few trillion. That sounds enormous, until you realise modern AI has already pretty much mined (read) all of it. After that point, there’ll be nothing new left to learn. Remember, Gen AI can not cure cancer, or create the schematics for a working warp drive, because humanity hasn’t discovered how to do that yet. Gen AI can only join the dots backwards, not forwards.

The hazard of AI is lazily allowing new culture to be generated rather than crafted. If everyone were to stop drawing and use Midjourney instead, there’d be no new images. If humanity were to cease writing, there’d be no new stories, and so nothing original on which to train our next generation of artificial ingenuities.

I’ve heard the argument there aren’t really any new stories anyway, and that every story is merely a derivative of a few prototypical tales — but that’s like dismissing the astounding diversity of life as just variations of bacteria, archaea, and eukaryotes.

New stories extend the possibility space for all humanity. New inventions reveal previously unseen truths. Each is the result of its author’s laborious lifelong quest to extract meaning from noise, to create grace out of entropy. Only creators and inventors join the dots forward, so others who follow can learn from them.

Gen AI is capable of style transfer, an ersatz form of creativity where new works can be synthesised “in the style of” a previously seen genre or creator. But AI does not possess the vital spark of ingenuity. Consider this: despite having memorised the entirety of published human knowledge, no AI model has ever produced a single noteworthy invention or scientific breakthrough.

AI models might seem like libraries of boundless knowledge, but actually they’re searchers just like us, albeit ones able to move far further and read much faster than we ever could. But models can only tread paths prior creators have illuminated, and which they’ve been trained on. The dark regions, still unlit by human intellect, will remain unknown until bold pioneers explore and explain them.

Once upon a time in Buenos Aires, whilst civilization around the globe burned to ashes in a brutal war, a nearly blind visionary imagined an infinite library of unfathomable vastness. It’s a paean to every human being who’s ever contributed an extraordinary treasure to its shelves, those whose own search for truth stocks the library with precious meanings.

The Library of Babel provides the answer to our riddle, and it turns out to be a paradox. Possessing infinite books does not grant us infinite knowledge. Because knowledge requires the exploration of discerning minds, and without them, an endless library is merely a lifeless tomb of unread books.

— Jaron 2023