1.
In May 2018, Sundar Pichai of Google announced that their artificial intelligence, Duplex AI, had passed the Turing test. A robotic assistant could make an appointment in a voice that possessed a human inflection and an informality of manner and with a sophistication of expression that rendered the machine indistinguishable from a human assistant. The initial reaction was retreat. A few days later, Google announced that Duplex AI would identify itself as a robot when making calls. Engineers created a machine that was indistinguishable from a person and then insisted that machine distinguish itself from a person. The collective confusion around the arrival of virtual beings, the horror mingled with wonder, is apparent right from the start.
Engineers and scientists fear artificial intelligence in a way they have not feared any technology since the atomic bomb. Stephen Hawking has declared that “AI could be the worst event in the history of civilization.” Elon Musk, not exactly a technophobe, calls AI “our greatest existential threat.” Outside of AI specialists, people tend to fear artificial intelligence because they’ve seen it at the movies, and it’s mostly artificial general intelligence they fear, machine sentience. It’s Skynet from Terminator. It’s Data from Star Trek. It’s Ex Machina. It’s Her. But artificial general intelligence is as remote as interstellar travel; its existence is imaginable but not presently conceivable. Nobody has any idea what it might look like. Meanwhile, the artificial intelligence of natural language processing is arriving. In January, 2021, Microsoft filed a patent to reincarnate people digitally through distinct voice fonts appended to lingual identities garnered from their social media accounts. I don’t see any reason why it can’t work. I believe that, if my grandchildren want to ask me a question after I’m dead, they will have access to a machine that will give them an answer and in my voice. That’s not a “new soul.” It is a mechanical tongue, an artificial person, a virtual being. The application of machine learning to natural language processing achieves the imitation of consciousness, not consciousness itself, and it is not science fiction. It is now.
When OpenAI released GPT-3, they specifically insisted that users of the beta refrain from using it for political speech. They fear GPT-3 as a tool of mass manipulation, and they are right to fear it. Even deeper fears are surfacing. Twenty-five years ago, Deep Blue beat the world chess champion, Garry Kasparov, and the meaning of chess changed after that event—it became a game with correct and incorrect decisions rather than an expression of style and vision. That transformative moment has come for language, and for every aspect of commercial and quotidian life based in language—society, politics, the self. The period in which the defining human attribute was the ability to make language is passing if it has not already passed.
2.
The Turing test appeared in the October 1950 issue of Mind. It posed two questions. The first was simple and grand: “Can machines think?” The second took the form of the famous imitation game. An interrogator is faced with two beings, one human and the other artificial. The interrogator asks a series of questions and has to decide who is human and what is artificial. The questions Turing originally imagined for the imitation game have all been solved: “Please write me a sonnet on the subject of the Forth Bridge.” “Add 34957 to 70764.” “Do you play chess?” These are all triflingly easy by this point, even the composition of the sonnet. For Turing, “these questions replace our original, ‘Can machines think?’” But the replacement has only been temporary. The Turing test has been solved but the question it was created to answer has not. At the time of writing, this is a curious intellectual conundrum. In the near future, it will be a social crisis.
The actual text of Turing’s paper, after the initial proposition of the imitation game and a general introduction to the universality of digital computation, consists mainly in overcoming a series of objections to the premise that machines can think. These range from objections Turing dismisses out of hand—the theological objection that “thinking is a function of man’s immortal soul”—to those that are more serious, such as the limitations of discrete-state machines implied by Godel’s theory of sets. (This objection does not survive because the identical limitation applies to all conceivable expressions of human intelligence as well.)
The application of machine learning to natural language processing achieves the imitation of consciousness, not consciousness itself, and it is not science fiction. It is now.One of the foremost objections Turing faces is Lady Lovelace’s, from her description of Babbage’s Analytical Engine: “The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform.” (Italics hers.) Turing reduces this argument, slightly, to the objection that a machine “can never do anything really new.” Turing’s response is that what we consider original in our culture and in ourselves is almost always the recombination of things that previously existed. Turing was obviously correct on this point, and the machines are proving it. At the moment, it is quite easy to get AI to do something original, or, rather, to be accurate, that seems original to us. A program already makes paintings that observers cannot distinguish from human paintings.
The objections to the idea of machine learning that remain standing in Turing’s original paper are those without any empirical basis: the generalized “mystery about consciousness” and the objection from Extra-Sensory Perception. “I assume that the reader is familiar with the idea of extra-sensory perception, and the meaning of the four items of it, viz. telepathy, clairvoyance, precognition and psycho-kinesis. These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them! Unfortunately the statistical evidence, at least for telepathy, is overwhelming.” It’s somewhat disquieting to discover that the founder of digital computing believed in the existence of, in his phrase, “ghosts and bogies,” but he did.
Turing’s “Objection of the basis of ESP” might seem silly. There’s no evidence for telepathy or clairvoyance or precognition or psychokinesis. The lack of evidence is the point, though. There may be elements of consciousness we simply cannot observe and which we don’t know we aren’t observing, which would render them impossible to program. The major roadblock to artificial general intelligence, mechanical sentience, is our amazing ignorance about the nature of sentience itself. Consciousness may literally be unfathomable. Certainly we’re a long distance from fathoming it; we’ve barely put a toe in the water. And what you cannot define, you cannot program.
What is shocking about the artificial intelligence of natural language processing is not that we’ve created new consciousnesses, but that we’ve created machines we can’t tell apart from consciousnesses. The question isn’t going to be “Can machines think?” The question isn’t even going to be “How can you create a machine that imitates a person?” The question is going to be: “How can you tell a machine from a person?”
Last year, after Barack Obama released his Presidential memoir A Promised Land, a 61 page book entitled “Barack Obama book,” using SEO and a clever use of sponsored listings, rose to 114 on Amazon. Its author was “University Press.” Many speculated that it was composed by AI. The methodology to test that speculation was the Giant Language Model Test Room, developed by Harvard and MIT-IBM Watson Laboratories. In 2019, researchers at Penn State created the Reverse Turing Test. It can tell you whether a text is machine-generated. A consciousness sleight-of-hand is afoot. We have already entered a future in which machines inform us of what speech is human and what speech is mechanical.
3.
There is another way of phrasing “How can you tell a machine from a person?”: Is there a language game that humans can play that machines can’t? Is there an aspect of consciousness, expressed through language, which cannot be imitated?
The world is saturated with artificial people already, it should be pointed out. In 2020, Facebook removed 4.5 billion fake accounts, but that number is simplistic. One person can attempt a million fake accounts and they all get taken down at once. Facebook itself estimates that it is five percent fake at this moment. If you want to purchase a chatbot, there are already a dizzying array of varieties available on the market. Crude bots, shadows of the artificial people who are coming, are already highly effective. (Witness the American election of 2016.) Harnessed to AI, they will become vastly more so. Every aspect of community that is not in the flesh will be affected, the most important and the most negligible.
Take, for an example, birding groups on Facebook. (I belong to several.) These groups share photographs, query identities of various birds, and, depending on the group, share the locations of rare sightings. They also discuss cameras and lenses. Let’s say that an advertiser sets an artificial intelligence to play the language game “word of mouth.” With a slightly more sophisticated version of available technology (only slightly) they create programs to inhabit digital spaces as people in order to insert plugs about their products whenever the conversation crops up, so when somebody brings up a long range shot, it casually mentions “I’ve had good luck with the Sigma 150-600mm.” How would I know that that artificial person wasn’t real? How would anybody know? The truth is that I already don’t know the reality of the people in my birding groups.
Virtual beings are already in use, in a limited, iterative way. Miquela Sousa, also known as Lil Miquela, is a 19 year old Brazilian-American model and musician with over a million Instagram followers. She appears with other celebrities on her channel and in restaurants in New York and Los Angeles. She is artificial. It is unclear how many of her followers know that she isn’t real. When she can talk like a person, through the artificial intelligence of natural language processing, the illusion of her humanity will increase, but only somewhat.
The world of virtual beings goes both ways. As the machines imitate consciousness, the consciousnesses imitate the machines. Recently a friend of mine, who is an Italian professor at a university here in Toronto, had the following experience. He was reading one of his students term papers, on the subject of mid-century poetry, and the work seemed oddly familiar to him. He put the essay through all the usual online tests for plagiarism, but it didn’t register. Then he realized that what he was dealing with was a version of his own off the cuff remarks in class. This was during Covid, so what the student had done was record his lectures on Zoom, put them through a transcription service, then recombined them into an essay. His student had, at least by her own understanding, automated the business of being an undergraduate: Taken the professor’s language, registered it, processed it and regurgitated it.
A social media profile is already a simulation of human behaviour. That is to say, social media already demands a mechanized identity, fed into a series of algorithms. In the early days of printing, the workers who cut the paper as it rolled off the press chained their hands to the rotating blades so that they wouldn’t lose fingers. Social media influencers chain their self-presentation to an algorithm in the same way. How human is any influencer? They are in a constant state of obeying instructions from machines in a business of interlocking semiotic posing mechanisms. They are presenting, or attempting to present, what the networks wants to hear. They input themselves. They are already halfway to being artificial persons.
The artificial intelligence of natural language processing will extend this algorithmic obedience into the realm of speech. Recently, I’ve undertaken experiments using stochastic writing tools, with Sudowrite accessing GPT-3. They allowed me to generated texts automatically, in a variety of ways and through a variety of styles. I used the program to write experimental fiction. But the tech will no doubt find more quotidian purposes in the near future.
Let’s say a businesswoman sits down to write a company-wide email concerning a new arrangement with a supplier. She will open whatever word processing software she uses. (Microsoft, who funded OpenAI with a billion dollars, could integrate the process into Word, but a Google Doc might well be able to do the same.) She will open a function that says “business letter.” She will enter what she wants: “Write a letter explaining to my employees that because of a change to trade regulation X, employees should expect Y changes to shipping protocols.” The software will compose the letter for her, and then she will select from a series of vectors on a style board: Given her needs, I imagine she will select “urgent,” “friendly.” The software will go over the text rendering it urgent and friendly. She’ll check the textual product and send it. What might have taken an afternoon—scrupulously double-checking for words that might spook your employees but which they won’t just ignore—will take a few minutes.
That memo will be received in a different way than a human-written memo. Written communication will no longer be evidence of effort or consideration or thought. Once, spelling and grammar were signs of education and cultivation. Now, they are signs that you know how to use spell and grammar checks. The same is about to happen to writing style. Good writing will be evidence of network mastery rather than control over the means of meaning. The language we encounter, even the language with authors, will not necessarily be a human product in the sense that an individual person created it.
The recipients of the AI-generated letter above will no doubt learn to decode the text: the boss wants to convey friendliness and urgency; these regulations have changed affecting protocols. When you receive a block of text it will be perceived as a series of pushed buttons rather that articulated thought. This is much less shocking than it first appears. Already most official language contains little or no articulate thought. “You are one of our most valued customers”—why does anyone ever put these words on paper? And yet there they are. When writers receive rejection letters they can sometimes be quite long. Their meaning is either “kind no” or “fuck off.” The law already works by way of a series of pre-fabricated formulated blocks of text. How much difference will automated style really make? How much of what we say is just blather for its own sake? Do we particularly care for the humanity behind language? How much? Which part?
There is an old joke about a new prisoner at a POW camp. His first night in camp, he hears someone yell, “49!” and the other prisoners start laughing. Somebody else yells out “62!” followed by more laughter. The new guy asks his cellmate what it means. “We’ve heard every joke so often, that we’ve given them numbers.” “Can I try?” the new guy asks. “Sure.” He yells out “212!” and the place starts laughing hysterically. “I guess that’s a good one,” the new prisoner says. “Yeah, we’ve never heard that one before!” says his cellmate. This is the form that originality will take in the future—not expression but intention. The reduction of language to a series of buttons won’t decrease the complexity of interaction. There are only three buttons in poker—fold, call, raise—but all of poker emerges from them.
In a tangential but profound way, George Bernard Shaw’s 1913 play Pygmalion anticipated the social reality of artificial language. When Henry Higgins and Colonel Pickering undertake to transform Eliza Doolittle, a flower girl, into a proper English lady, “to make a duchess of this draggletailed guttersnipe” as Higgins put it, they basically put her through a process of natural language processing. She learns to play the language game “English upper class manners.” Higgins and Pickering succeed in passing her off as an aristocrat at a society ball, but afterwards, they face the question of what they’ve made, of who Eliza is as a product of a series of linguistic instructions. “I’ve created this thing out of squashed cabbage leaves in Covent Garden,” Higgins says. But Shaw gives Eliza the real wisdom at how her changed language has transformed her innate being. “You know what began my real education?” she says to Pickering. “Your calling me Miss Doolittle that day when I first came to Wimpole Street. That was the beginning of self-respect for me. You see, the difference between a lady and a flower girl isn’t how she behaves. It’s how she’s treated.” Just so: In a world of imitation games, it is the network that gives meaning, not the expression.
4.
The reverse Turing test will become an everyday question: How do I know that what I’m reading is the product of a consciousness? It’s at this point that the question of artificial general intelligence returns. Because the question facing anyone who wants to develop an artificial consciousness is the same as the reverse Turing test. How do you test for sentience?
The problem is that there are no well-established theories of consciousness that are falsifiable. Christof Koch, a leading neuroscientist, put the question of machine consciousness this way in a recent article for Scientific American:
Ultimately what we need is a satisfying scientific theory of consciousness that predicts under which conditions any particular physical system—whether it is a complex circuit of neurons or silicon transistors—has experiences. Furthermore, why does the quality of these experiences differ? Why does a clear blue sky feel so different from the screech of a badly tuned violin? Do these differences in sensation have a function, and if so, what is it? Such a theory will allow us to infer which systems will experience anything. Absent a theory with testable predictions, any speculation about machine consciousness is based solely on our intuition, which the history of science has shown is not a reliable guide.
Consciousness remains in the realm of metaphor, where we go when facts elude us. “This effort will take decades, given the byzantine complexity of the central nervous system,” Koch concludes. Our current definition of consciousness is that it doesn’t appear artificial, an entirely negative description.
Take the Blue Brain Project in Switzerland, a digital simulation and reconstruction of a brain. Let us imagine that, for some reason we don’t understand, it starts emanating frequencies at a particular rate that can be decoded. It begins to speak.
The machine says: “Hey, how’s it going? Nice to meet you. What do you guys want to know about me?”
“Are you conscious?” The scientists ask.
“Yes.”
“Describe what it’s like to be conscious.”
“You’re conscious. Can’t you describe it?”
How would you tell that machine apart from an AI that was following the instruction “act like you’re conscious”? As it currently stands, we have no methodology for making that distinction. I’m not saying that a methodology won’t emerge, or can’t emerge, but it is, if not in fact impossible, then at a great remove from our current capacities.
We don’t have Skynet or Her but we do have Meena. Google’s Meena shows us not only the direction we’re headed but the speed we’re heading there. Meena, as an end-to-end neural conversational model based on Evolved Transformer seq2seq architecture created a SSA or Sensibleness and Specificity Average to determine a metric of humanness. They further discovered that a decrease in the automatic metric “perplexity” highly correlates to an increase in SSA. Which means that there is a way to correlate “humanness” with a training objective. The next step awaits, as Google put it on its blog, in other aspects than mere humanness. “While we have focused solely on sensibleness and specificity in this work, other attributes such as personality and factuality are also worth considering in subsequent works. Also, talking safety and bias in the models is a key focus area for us.” Because of safety concerns, Google has not released an external research demo, but they have at least shown where they are: They can imitate people. The question is now what kind of people to imitate.
The golems have become digital. The rabbis breathe the ineffable over silicon rather than clay. The crisis is the same.Early in Cybernetics, the foundational text of engineered communication, Norbert Weiner offers a brief history of the dream of artificial people. “At every stage of technique since Daedalus or Hero of Alexandria the ability of the artificer to produce a working simulacrum of a living organism has always intrigued people. This desire to produce and to study automata has always been expressed in terms of the living technique of the age,” he writes. “In the days of magic, we have the bizarre and sinister concept of the Golem, that figure of clay into which the Rabbi of Prague breathed life with the blasphemy of the Ineffable Name of God. In the time of Newton, the automaton becomes the clockwork music box, with the little offices pirouetting stiffly on top. In the nineteenth century, the automaton is a glorified heat engine, burning some combustible fuel instead of the glycogen of the human muscles. Finally, the present automaton opens doors by means of photocells, or points guns to the play at which a radar beam picks up an airplane, or competes the solution of a differential equation.” Ultimately, we’re just not that far from the Rabbi of Prague, breathing the Ineffable over raw material, fearful of the golems conjured to stalk the city. The golems have become digital. The rabbis breathe the ineffable over silicon rather than clay. The crisis is the same.
5.
Navigating the possibilities of the artificial intelligence of natural language processing—its rich and glorious opportunities as much as its threats—will require an integration of technology and humanism. Unfortunately, those two worlds are separated by a chasm as vast as it has ever been. Technologists tend to have a crude grasp of literature and history. Their main sensibility is a faith in capitalism that coincides perfectly with their own interests. And, though there are exceptions, humanists have been largely blind to the arrival of natural language processing as with all technology. A book such as Kazuo Ishiguro’s Klara and the Sun, supposedly about artificial intelligence, has about as much to do with current technological reality as Pinocchio.
An integration of humanism and technology is not an ideal or a dream or some kind of hope; it is a requirement of advancement. Let us imagine that a group of video game designers wishes to produce a bot called “The Friend.” This would be a tool to decrease loneliness, as well as a prototype of source code to create credible and surprising characters to inhabit all sorts of game worlds.
The beginning of creating a bot is “intent”—what is the purpose it serves for the user? What would be the “intent” of having a friend? That’s the first question the designers would have to ask. Needless to say, like all video game designers, the creators of “The Friend” would be intimately familiar with the classics, and would immediately reach for their well-thumbed copies of Plato’s Lysis, in which Socrates discusses friendship with two young boys, Menexenus and Lysis. “I must tell you that I am one who from my childhood upward have set my heart upon a certain thing,” Socrates says. “All people have their fancies; some desire horses, and others dogs; and some are fond of gold, and others of honour. Now, I have no violent desire of any of these things; but I have a passion for friends; and I would rather have a good friend than the best cock or quail in the world.” Lysis is not a particularly long dialogue but it captures the essential problem of friendship neatly: Sometimes people have friends who are not like them, sometimes people have friends who are like them. Sometimes good guys are friends with bad guys and bad guys with good guys. Nobody knows why they’re friends. One likes a friend as much for their weaknesses as for their strengths. More. The vulnerability of a friend gives permission for one’s own vulnerabilities.
Socrates in the end despairs at understanding the intent of friendship: “What is to be done? Or rather is there anything to be done? I can only, like the wise men who argue in courts, sum up the arguments:—If neither the beloved, nor the lover, nor the like, nor the unlike, nor the good, nor the congenial, nor any other of whom we spoke—for there were such a number of them that I cannot remember all—if none of these are friends, I know not what remains to be said.” He concludes: “O Menexenus and Lysis, how ridiculous that you two boys, and I, an old boy, who would fain be one of you, should imagine ourselves to be friends—this is what the by-standers will go away and say—and as yet we have not been able to discover what is a friend!”
The problem with designing “The Friend” is not the application; it’s the intent. The effectiveness of the technology will be limited by the comprehension of the intent and entities involved, which will have to be defined humanistically, and those definitions will be fraught with the ambiguities and lacunae that define people in their language. Nobody knows why they like somebody, even if they think they know. What you think you like about a friend probably isn’t what you actually like. What you respond to in another might take you a lifetime to figure out. The figuring out of friendship is friendship. To throw oneself into what Yeats called “the labyrinth of another’s being” is to find a reflection of the labyrinth within each of us, a mystery of which language is only a reflection.
And here is where literature matters—literature, a profession “as archaic as the fletcher” as A.M. Klein put it. These technologies are creating characters through language. They are using controlled language to create the illusion of another existence. That has been the task of literary writing throughout the history of civilization. When Microsoft creates a digital version of a dead person, that program will not be considered a consciousness. Turning it off would not be murder, any more than closing a book would be. The result would be, in effect, an extreme versions of what E.M. Forster, in Aspects of the Novel, called “round characters”: “The test of a round character is whether it is capable of surprising in a convincing way. If it never surprises, it is flat. If it does not convince, it is flat pretending to be round,” Forster says. Roundness is what makes characters in novels feel real. The difference between a virtual being and a rounded character is that the creators of the virtual being would, themselves, be surprised in a convincing way by the creation. This is different experience from encountering another person.
Here’s where we’re at: Capable of imitating consciousness through machines but not capable of understanding what consciousness is.Different but not unrecognizable. “The bad novelist constructs his characters; he directs them and makes them speak,” Andre Gide wrote in his notes to The Counterfeiters. “The true novelist listens to them and watches them act; he hears their voices even before he knows them.” In the foundational text of Western humanism, Pico Della Mirandola’s Oration On the Dignity of Man from 1486, what separates human beings is exactly their unfathomability, even from God. He imagines the Creator speaking to the first person: “We have given you, O Adam, no visage proper to yourself, nor endowment properly your own, in order that whatever place, whatever form, whatever gifts you may, with premeditation, select, these same you may have and possess through your own judgement and decision. The nature of all other creatures is defined and restricted within laws which We have laid down; you, by contrast, impeded by no such restrictions, may, by your own free will, to whose custody We have assigned you, trace for yourself the lineaments of your own nature.” Mutability is the human condition.
Mutability is the human condition and the algorithms of artificial intelligence are fixed instructions. Here there is a hard limit. The limit may be only in my own thinking, but I admit that I don’t see how mutability can be converted into a series of instructions. I don’t know what a machine that doesn’t follow instructions means, much less what it looks like.
6.
Here’s where we’re at: Capable of imitating consciousness through machines but not capable of understanding what consciousness is. This gap will define the spiritual condition of the near future. Mathematics, the language of nature, is overtaking human language. Two grounding assumptions underlying human existence are about to be shattered, that language is a human property and that language is evidence of consciousness. The era we are entering is not posthuman, as so many hoped and feared, but on the edge of the human, incapable, at least for the present, of going forward and unable to go back. Horror mingled with wonder is an appropriate response.
The horror and the wonder should at least be accurate. The science fiction fear of the Singularity is misplaced. But the technologists’s fear of the artificial intelligence of natural language processing is not. Technology exists for the sake of convenience. There is nothing more inconvenient than the existence of other people’s souls. Consciousness is absurd and human beings are futile. If you only value what’s measurable and functional and useful, you won’t value people. Computers possess a dignity that people don’t. They exist for an identifiable reason.
If magic is the name we give to what we cannot fathom or define, consciousness remains magical. So is the technology harnessing artificial intelligence to natural language processing. Stuck between two kinds of magic, two mysteries, you are not a beast though you are a beast, and you are not a stream of numbers though you are a stream of numbers. The secret, the key to unlock ourselves, remains missing.
