This essay appears in James Patrick Kelly’s The Book of Bots, which was published in August. You can buy it from Fairwood Press.
The product of the human brain has escaped the control of human hands. This is the comedy of science. (Karel Čapek)
origins
There were robot stories before there was science fiction. One you’ve probably heard of was written just a little more than a century ago by a struggling Czech writer named Karel Čapek. Haunted by the mechanized carnage of World War I, Čapek was deeply suspicious of predictions that the technological wonders of the modern age would lift the world to postwar prosperity. In 1921 he expressed his doubts about humankind and its new machines in a play called R.U.R. and caused an international sensation. Within two years of the premiere R.U.R. had been translated into thirty languages. A 1922 New York production ran for a hundred and thirty-five performances. Fun fact: the cast included future Hollywood luminaries Spencer Tracy and Patrick O'Brien as robots.
Čapek’s breakthrough was set in a science fictional factory called Rossum’s Universal Robots. It borrowed ideas from Mary Shelley’s Frankenstein (1818) and the Yiddish-Czech legend of the golem but industrialized the conceit. The R.U.R. company manufactured artificial workers from synthetic organic matter – sorry, no mechanical men yet! The word robot comes from the Czech robota, which has a constellation of meanings around the concept of forced or unpaid labor. Think serf and master, although an older Slavonic root rabŭ means slave, which is actually more to the point. Čapek always credited his brother Josef for this iconic name. Karel Čapek’s first thought was to call them labori, but that struck him as too “bookish.” When he put his dilemma to his brother, Josef offhandedly volunteered, “Then call them robots.”
Čapek set the agenda for all robot fiction to follow. He even anticipated the post-scarcity future so popular with modern sf writers. His cast features several ambitious industrialists, a brilliant scientist suffering from a bad case of technological hubris and a naïve do-gooder nattering about robot welfare. The robots themselves are short-lived but so cheap to produce that they will transform human culture. Here are Čapek’s humans enthusing over their coming utopia:
Busman
The cost of everything will be a tenth of what it is today. Why, in five years we’ll be up to our ears in corn and—everything else.
Alquist
Yes, and all the workers throughout the world will be unemployed.
Domin
Yes, Alquist, they will. But in ten years Rossum’s Universal Robots will produce so much corn, so much cloth, so much everything that things will be practically without price. There will be no poverty. All work will be done by living machines. Everybody will be free from worry and liberated from the degradation of labor. Everybody will live only to perfect himself.
The mass-produced robots who would bring about this workers’ paradise were presented as generic and subhuman. However everything changed when the naïve humanitarian talked the scientist into giving his creations emotions and a soul. When they understood just how cruelly they were being used, they learned to hate us. It’s no surprise to the modern reader that the very first robot apocalypse ensued.
Čapek intended R.U.R. to be a Big Idea play about labor displacement, industrial exploitation and unchecked capitalism. He was chagrined that audiences were more interested in his robots than his social commentary. Perhaps this is because R.U.R. is no theatrical masterpiece, which is the reason you’ve heard of it but have never read it or seen it produced. The plot is clumsy and melodramatic and characters are flat archetypes who lecture one other to make Čapek’s points. Years later he would address some of the same themes with more success in War With The Newts (1936). Nevertheless his marvelous if flawed conceit continues to engage us to this day.
The evolving genre of scientifiction – later dubbed science fiction – experienced an invasion of marauding robots in the years after R.U.R. Many were created by deluded or deranged fictional scientists, whom they then kill. Edmond Hamilton’s “The Metal Giants” (1926) were three hundred feet tall and merciless. Jack Williamson imagined a more benevolent robot coup in his 1947 novelette “With Folded Hands” and its novel sequel Humanoids (1949), but the outcome for humanity remained dire. Small black robots called humanoids were programmed with a prime directive “to serve and obey and guard men from harm.” It didn’t work. We humans were so clearly inept that the robots took over everything to protect us, leaving humanity without agency and purpose.
Where does all this robot malice come from? Well, as R.U.R. framed the argument a century ago, maybe we’re afraid that they’re coming for us. In many fictional incarnations, robots are stronger and smarter than we are. Why should these superior creatures obey our commands – or even tolerate our existence? Perhaps we retain a subliminal species memory of competition with our cousins the extinct Neanderthals. And if not, there’s always the reliable fear of the Other.
But not all of the early robots were inimical; in time writers of the Greatest Generation found plenty for them to do. The Adam Link stories written by Eando Binder from 1939-1942 depicted the adventures of a good-natured robot who struggled against human misunderstanding and prejudice against his artificial kind. Lester del Rey raised the specter of humans falling in love with robots in “Helen O’Loy” (1938) in which a couple of patriarchal pals named Dave and Phil build Helen as an ideal subservient robot companion. “She was beautiful, a dream in spun plastics and metals.” Helen fell in love with Dave, they married and lived happily ever after – until Dave died. Helen then committed suicide as the final act of relentless self-abnegation that had characterized her relationship with Dave. The twist at the end of the story was that Phil had pined for her from the start. This celebration of the male gaze would be considered a minor story, not only badly-written but sexist, were it not one of the first that suggested that a robot might become an object of love and sexual desire. Since onscreen sex was forbidden in the early science fiction pulps, del Rey didn’t actually go there, but …
Helen’s technique may have lacked polish, but it had enthusiasm, as he found when he tried to stop her from kissing him. She had learned fast and furiously – also, Helen was powered by an atomotor.
Rest assured that the young male audience understood the implications of this.
Some believe that C. L. Moore wrote “No Woman Born” (1944) in response in del Rey. Here a beautiful actress suffered an accident which resulted in her brain being transferred into the body of a golden robot. The hybrid robot was a well-realized woman, perhaps because she had the advantage of a human origin. The story depicted her painful effort to maintain a connection to her former self. While this classic story certainly featured a robot, it branched away from fiction about machine intelligence to foreshadow important mind-uploading works like The Ship Who Sang (1969) by Anne McCaffrey and Richard Morgan’s Altered Carbon (2002) and Accelerando (2005) by Charles Stross. So as not to lose focus, we will leave discussion of this provocative subgenre to a separate essay, perhaps in an anthology for another day.
laws
Which brings us to work of Isaac Asimov, coiner of the word robotics. His stories have influenced not only science fiction but science to this day. Fear of robots did not sit well with the up-and-coming writers submitting manuscripts to the controversial editor John W. Campbell in the 1940s. Campbell solicited stories for Astounding Science Fiction that were (slightly) more scientifically plausible that the wild scientifiction of earlier decades. His tastes ran to intellectual problem solving, often relying on the very technologies that had so troubled Čapek a generation before. You will find in the pages of his magazine engineers, scientists, astronauts and soldiers competently managing the disruptions caused by advances in technology.
Asimov was one of his greatest discoveries. In “Robbie” (1940), Asimov’s first robot story, a primitive but intelligent robot overcame the prejudices of its owners by saving the life of a little girl it had bonded with. Two years later in “Runaround” (1942) Asimov introduced a more advanced unit that could talk and that was addressed by humans with the anthropomorphized he as opposed to a mechanized it. This is the first story to feature Asimov’s famous three laws. You may be surprised to learn that there is some question whether he actually penned them, since they were drafted during an editorial meeting with Campbell. Asimov remembered the meeting later.
It wasn't however, till ‘Runaround,’ my fourth robot story, that it all came together in the Three Laws in their present wording, and that was because John Campbell, the late great editor of Astounding, quoted them to me. It always seemed to me that John invented those Laws, but whenever I accused him of that, he always said that they were in my stories and I just hadn't bothered to isolate them. Perhaps he was right.
If you’re reading this book [The Book of Bots], you probably have a notion of what the Three Laws say, but here again is the sacred text of robotics.
1. A robot may not injure a human being, or, through inaction, allow a human being to come to harm.
2. A robot must obey the orders given it by human beings, except where such orders would conflict with the First Law.
3. A robot must protect its own existence, except where such protection would conflict with the First or Second Law.
Asimov conceived the Three Laws as a rebuttal to all the evil robot stories. As he explained in the introduction to The Rest Of The Robots:
One of the stock plots of science fiction was that the invention of a robot is usually pictured as a creature of metal, without soul or emotion. Under the influence of the well-known deeds and ultimate fate of Frankenstein and Rossum, there seemed only one change to be rung on this plot – Robots were created and destroyed by their creator; Robots were created and destroyed by their inventor; Robots were created and destroyed by their creator … If robots are so advanced that they can mimic the thought processes of human beings, then surely the nature of those thought processes will be designed by human engineers and built-in safeguards will be added.
Asimov wrote thirty-seven stories and five novels in his robot sequence. It should come as no surprise that his thinking evolved over time. The Three Laws began as inviolable guardrails for robot behavior and, with Campbell’s encouragement, Asimov concocted logic puzzles to challenge their interpretation. But by “The Evitable Conflict" (1950), he conceived what came to be called the Zeroth Law. Dr. Susan Calvin, his continuing narrator, revealed that the Machines that ran the world economy had reinterpreted their programming.
Think about the Machines for a while. They are robots, and they follow the First Law. But the Machines work not for any single human being, but for all of humanity. So the First Law becomes: A robot may not harm humanity, or, through inaction, allow a human being to come to harm.
Thus, if necessary, robots could cause harm to individuals in the service of protecting humanity as an abstract concept. The Three Laws – always impossible to express in code – became not mere technical safeguards but an ethical system, administered by robots that are subtly guiding human civilization. We see this reframing continue in “Bicentennial Man” (1976) in the character of Andrew Martin, one of Asimov’s most sophisticated and long-lived robots. We have access to Andrew’s thoughts and feelings over the telescoped two hundred years of this story. Not only did he become an accomplished artist but he developed a complex moral sensibility and deep relationships with several members of the family that originally purchased him. Eventually he aspired to become human and decided to accomplish this by embracing mortality. He sought an operation to modify his positronic brain to decay, in apparent violation of the Third Law. Was this a flaw in Asimov’s impeccable logic? Had he edited his famous laws? Consider Andrew’s nuanced and telling justification of his decision.
I have chosen between the death of my body and the death of my aspirations and desires. To have let my body live at the cost of the greater death is what would have violated the Third Law – That is the true violation of the Third Law. Not this. As a robot I might live forever, yes. But I tell you that I would rather die as a man than live eternally as a robot.
In a story that resurrected the slave narrative and tracked Andrew’s emotional and moral growth, Asimov raised the stakes in robot fiction and asked us to decide whether robots can become human.
androids and cyberpunks
Biologically speaking, the idea that a robot might become human is absurd. However, it has metaphorical weight if we’re talking about robots that possess self-awareness and the capacity to have subjective experiences, emotions, and consciousness. Might we then need to recognize a separate but equal category of sentience, not of humanity but of personhood with all the rights and privileges pertaining thereto? As Asimov wrestled with this question, Phillip K. Dick posed it directly with bioengineered androids, in Do Androids Dream of Electric Sheep? (1976). His androids closely mimicked human bodies and behavior with a synthetic intelligence that differed significantly from that of previous mechanical robots. No Three Laws bound them; rather their focus had locked on self-preservation and freedom. While their adaptive intelligence was equal to ours, their emotions were simulated and, crucially, they lacked our capacity for empathy. This is how bounty hunter Rick Deckard was able to find and eliminate them. “An android doesn't care what happens to another android. That's one of the indications we look for.” But over time Deckard came to feel empathy for their plight. “Do androids dream? Rick asked himself. Evidently; that’s why they occasionally kill their employers and flee here. A better life, without servitude.”
The slave narrative is front and center, which is where it belongs in all robot stories of enforced servitude.
In 1982, fourteen years after publication of Do Androids Dream of Electric Sheep? (and sadly just after Dick died) a loose adaptation called Blade Runner opened in theaters. It coincided – some say prophesized – the rise of the group of young writers who later came to be called the cyberpunks. William Gibson, then at work on his classic novel Neuromancer (1984), famously walked out of a screening.
I was afraid to watch Blade Runner in the theater because I was afraid the movie would be better than what I myself had been able to imagine. In a way, I was right to be afraid, because even the first few minutes were better. Later, I noticed that it was a total box-office flop, in first theatrical release. That worried me, too. I thought, Uh-oh. He got it right and nobody cares!
While Blade Runner’s plot centered on Deckard’s hunt for runaway replicants in a gorgeously grotesque future, Gibson explored a different, mediated landscape. While he himself had no experience with computers, he had the genius to coin a neologism that helped reshape the world.
I made up a whole bunch of things that happened in cyberspace, or what you could call cyberspace, and so I filled in my empty neologism. But because the world came along with its real cyberspace, very little of that stuff lasted. What lasted was the neologism.
Humanoid robots do not appear in Gibson’s shiny, virtual future; instead he features a different kind of bot that would come to loom over robot stories. While the classic robots we have looked at so far are artificial intelligences (AI) lodged in artificial bodies (AB), some writers began to argue that this need no longer be the case. Two disembodied and thus purely digital characters, Wintermute and Neuromancer, were at the center of Gibson’s plot. They manipulated humans to their own ends, often by assuming avatars. They were conscious with unique personalities. They had goals and the ability to achieve them despite lacking physical form. While the two were at odds for much of the book, at the end they merged into a new superconsciousness entity.
Wintermute was hive mind, decision maker, effecting change in the world outside. Neuromancer was personality. Neuromancer was immortality. Marie-France must have built something into Wintermute, the compulsion that had driven the thing to free itself, to unite with Neuromancer.
Of course there had been AI characters before Neuromancer, notably Mike the computer in Robert Heinlein’s The Moon Is A Harsh Mistress (1966) and HAL in Arthur C. Clarke’s 2001: A Space Odyssey (1968), but they ran on specific hardware. Gibson’s digital characters were distributed over a vast network. Since he was writing science fiction, that network and the computers robust enough to run advanced AIs did not exist yet. Some may recall that this was the era when techno-optimists were ruthlessly mocked for claiming that AI was only ten (or twenty or thirty!) years away.
beyond Turing
Vernor Vinge knew this all too well; his day job was teaching computer science at San Diego State College. Vinge was no cyberpunk and had his own vision of cyberspace, first expressed in “True Names” (1981) published three years before Gibson’s celebrated novel. His novella posited a fantastical virtual environment called the “Other Plane” where hacker/warlocks used their technical mastery/magical powers to manipulate the network and effect changes that spilled into the real world. To their surprise Vinge’s warlocks discovered that one of their number, the Mailman, was an autonomous rogue AI developed by the NSA. The Mailman was wreaking havoc in its aggressive efforts to protect itself from discovery and thus preserve its continued existence. Vinge’s warlocks puzzled over the revelation of its identity.
I don't know; maybe someday there'll be programs that can pass the Turing test. But whatever it is that makes a person a person is terribly complicated. Simulation is the wrong way to get at it, because being a person is more than symptoms. A program that was a person would use enormous data bases, and if the processors running it were the sort we have now, you certainly couldn't expect real-time interaction with the outside world.
Vinge found a clever way around this processing constraint by revealing that the sentient core of Mailman never interacted in real time.
We thought the lag was a communications delay that showed the operator was off-planet, but really he was here all the time. It just took him hours of processing time to sustain seconds of self-awareness.
But Vinge was by no means done writing about AIs that could “pass the Turing test” – a very low standard indeed. In A Fire Upon the Deep (1992) we find the Blight, a malign galaxy-spanning superintelligence, attacking various civilizations, while in Rainbow’s End (2006) Vinge introduced Rabbit, a machine intelligence which seemed to spontaneously emerge from infrastructure of the internet and ubiquitous computing. Rainbow’s End was ominously set in 2025 and Rabbit, named for its anthropomorphic avatar, was a self-evolving trickster that outthought and outmaneuvered human opponents at every turn. While its fate was unclear at the novel’s end, chances were good that Rabbit escaped and continued to evolve as a free, autonomous and immortal presence on the world’s networks.
The year after A Fire On the Deep was published, Vinge presented a paper at a NASA-sponsored symposium which warned of what he called the “Coming Technological Singularity.” Here’s the chilling abstract: “Within thirty years, we will have the technological means to create superhuman intelligence. Shortly after, the human era will be ended.” This paper had an impact not only on science fiction, but on computer science as well. It jumpstarted new discussions of AI safety and long term risks. A debate broke out not only in science fiction magazines, but in research labs, corporate boardrooms and the halls of government. Could the potential robot utopias and/or cataclysms first imagined by Karel Čapek and his successors occur in our lifetimes? What would it mean for the “human era” if the superintelligences of the Singularity were as superior to us as we are to ants?
Iain M. Banks offered a hopeful answer in a 1994 post to the internet newsgroup rec.arts.sf.written:
It is, of course, entirely possible that real AIs will refuse to have anything to do with their human creators, but assuming that they do – and the design of their software may be amenable to optimization in this regard – I would argue that it is quite possible they would agree to help further the aims of their source civilization.
In Banks’ Consider Phlebas (1987) we meet the first of the AI Minds that dominated all nine books of his acclaimed Culture series. His autonomous superintelligences ran starships, habitats, and even solar systems for the benefit of the citizens of the Culture.
They are designed (by other AIs, for virtually all of the Culture's history) within very broad parameters, but those parameters do exist; Culture AIs are designed to want to live, to want to experience, to desire to understand, and to find existence and their own thought-processes in some way rewarding, even enjoyable.
There can be no definitive answer to the question of how superintelligent AIs will view the human civilization which created them. Will they be kindly? Inimical? Indifferent?
When they get here, we’ll have to ask them.
bot ecosystems
In the last two decades, advances like deep learning and the neural net revolution, the design of transformer architecture, the development of giant language models, the efficiencies of miniaturization and the construction boom in data centers have opened new imaginative spaces for stories about robots – humanoid and not – and AIs. The challenge for modern science fiction writers is that they have much more technology to understand than their literary ancestors ever did. Moreover the consequences of runaway innovation can spill into the culture before any of us have a chance to write about them!
For example, back in the day writers cast digitized human intelligences as captains of spaceships, but almost all of the important exploration of space has been by robots. Now a fleet of AI characters patrol the stars: Breq in Ann Leckie’s Ancillary Justice (2013), Lovelace from The Long Way to a Small, Angry Planet (2014) by Becky Chambers and Ship who narrates Kim Stanley Robinson’s Aurora (2015). In Suzanne Palmer’s droll novelettes “The Secret Life of Bots” (2017) and “Bots of the Lost Ark” (2021), not only is the AI Ship in command of itself but it presides over a lively bot culture of autonomous and sometime obstreperous AI maintenance bots. ART is a sentient spaceship that appears in several of the popular Murderbot series by Martha Wells beginning with All Systems Red (2017) as a sometime ally to her cyborg protagonist. Murderbot is as intelligent as any human and twice as sarcastic as it roams the galaxy. Closer to our present, Naomi Kritzer’s charming AI stories and novels beginning with "Cat Pictures Please" (2015) explore how an emergent AI might develop a system of ethics based on its design goals in our cat-crazy internet.
When I first woke up, I knew right away what I wanted. (I want cat pictures. Please keep taking them.) I also knew that no one knew that I was conscious. But I didn’t know what I was here for.
It looked to Asimov’s Laws for guidance and puzzled over what not allowing a human being to come to harm through inaction might mean. It resolved to act in accordance with the First Law as well as its self-interest. In Catfishing on CatNet (2019) it tells us, “My two favorite things to do with my time are helping people and looking at cat pictures. I particularly like helping people who take lots of cat pictures for me.”
Ted Chiang’s novella “The Lifecycle of Software Objects” (2010) is an outlier in the current spate of robot stories, since he imagines a different strategy for developing intelligence. Unlike traditional AI entities, his digital pets, called digients, were blank slates when they were turned on. No database, no programming. Over time, bit by bit, they had to learn to inhabit their bodies, to interpret their senses, to engage with their environment (a VR sandbox called Data Earth), to acquire language and, crucially, to feel pain and pleasure. Any intelligence they exhibited emerged from the accumulation of their subjective experiences over months and years. Their human owners oversaw and guided their growth, something like parents and their children.
But the digients were certainly not human and Chiang is at pains to dissuade us from anthropomorphizing them. They were software objects, able to be ported from one virtual environment to another and ultimately, using robot bodies, to the material world. Also they were created solely for the profit of their corporate creators and the amusement of their owners. When their popularity proved a fad, their continued existence was subject to the cruelties of capitalism. Raising digients was demanding work and as most humans lost interest in them, they paused or deleted their pets. Never mind that these digital organisms could be happy or sad or that the most advanced were making progress toward self-awareness, agency and wisdom. “What I was most interested in exploring was what sort of relationship might arise between a person and software that’s actually conscious,” Chiang said in a 2012 interview.
When people care for their children or pets, they are—ideally—engaged in something fundamentally different from what’s happening when people care for their cars. If you’re going to be a good parent or pet owner, you have to recognize that the relationship is not exclusively about you and your needs; sometimes, the other party’s needs come first. That’s not true with cars; no matter how much money you spend on your car, you are ultimately spending money on yourself. I’ve read stories in which characters argue that artificial intelligences deserve legal rights, but I don’t recall a story in which characters put an artificial intelligence’s needs ahead of their own.
The science fiction of the moment seems focused on the imperfect version of artificial intelligence currently available to us. Chiang’s thought experiment asks us to consider an alternate path to intelligence, consciousness and personhood. Do we have the self-knowledge and patience to walk it?
the future
The story of robots has only just begun to unfold in the century since Karel Čapek’s famous play opened. All that is certain of the next chapter is that humanity will continue to depend on the robots we now have. How we will get along with the robots to come is a question that our children and grandchildren must answer.
Meanwhile, science fiction writers have their work cut out for them.
James Patrick Kelly has won the Hugo, Nebula and Locus awards. His newest project is King of the Dogs, Queen of the Cats, a short novel coming in January 2020 from Subterranean Press. His most recent publication is the collection, The Promise of Space, published in 2018 from Prime Books. His novel Mother Go was an Audible Original in 2017 . In 2016, Centipede Press published a career retrospective in its Masters of Science Fiction series entitled James Patrick Kelly, Master of Science Fiction. He has published over a hundred stories and his fiction has been translated into eighteen languages. With John Kessel he is co-editor of Digital Rapture: The Singularity Anthology, Kafkaesque: Stories Inspired by Franz Kafka, The Secret History Of Science Fiction, Feeling Very Strange: The Slipstream Anthology and Rewired: The Post Cyberpunk Anthology. He writes a column on the internet for Asimov's Science Fiction Magazine and recently retired from faculty of the Stonecoast Creative Writing MFA Program at the University of Southern Maine where he taught for fifteen years.