Ride or die.
Support The Bulwark.
  Join Now

Making Machines, Remaking Ourselves

The brain enhancements and robotic friends of Kazuo Ishiguro’s ‘Klara and the Sun.’
April 17, 2021
Featured Image

Klara and the Sun
by Kazuo Ishiguro
Knopf, 303 pp., $28

Kazuo Ishiguro’s new novel, Klara and the Sun, is in many ways a revisiting and extension of the Genesis story of humanity’s creation and fall in the Garden of Eden. The new original sin is the similar to the first: the grasping of knowledge that is not good for us to have. In this case, the forbidden fruit is the capacity to instantiate and manipulate intelligence. Klara is a story of artificial intelligence, but not the kind runs on circuit boards. Rather, the story runs through the human heart—alerting us to the cascade of consequences that might result by alterations of human intelligence in our ever-accelerating race to gain advantage over, or at least keep up with, one another. In this book, AI is merely a foil for pointing out to the reader all that we risk losing by an effort to seize control of the commanding depths, buried deep in our DNA, of human intellect.

Ishiguro tells moral stories without ever drifting into moralism proper. His most famous novel, The Remains of the Day (1989), tells the story of an English butler so lost in the excellence of his craft that he insensible either to its moral dangers or how it is blinding him to what is his last, and perhaps his only, chance at finding love. In Never Let Me Go (2005), Ishiguro uses biotechnology as the bleak backdrop for a sensitive, subtle story about beauty, sacrifice, and hope. He doesn’t judge his subjects; he lets his characters judge their lives in moments of excruciating self-realization. Human beings are often a great disappointment—to themselves.

Ishiguro is not moralizing in Klara either but is suggesting that heartrending regret may be involved in humanity’s drive toward biotechnological self-improvement. Ishiguro knows the competitive nature of human beings; the drive to excel and the imperative to seek advantage for ourselves and our offspring may make it all but impossible to resist new biotechnological techniques that promise the improvement or “perfection” of our bodies and minds. People willing to engage in cutthroat competition for slots in preferred daycares and preschools will not long resist adding to a child’s store of intellectual ability by nipping and tucking material that is invisible without an electron microscope.

What he does seem to suggest is that by doing so we invite the mother of all knowledge problems. In the novel, the rise of a technique that can “lift” intelligence and ability is a dice-roll that, in some cases, becomes a death sentence. Whatever level this technique works at (genes, neurons, something else), it occasionally interacts with the body in unexpected and dangerous ways. Without complete knowledge of the interrelationships among genes or brain structures—at the level of individuals, not just populations—we cannot be sure that by adding or subtracting we aren’t wreaking havoc in ways that are at once invisible, momentous, and potentially deadly.

Our protagonist, Klara, is an “artificial friend” or AF, a type of robot that provides companionship and social-emotional training for the lifted. Josie, a lifted teen, chooses and, remarkably, is chosen by, Klara. Josie is also suffering a mysterious degenerative disease brought on by her lifting. All lifted children pass through this phase but not all survive it. What neither Klara nor Josie is aware of is that for Josie’s mother, Klara is an insurance policy. The mother’s hope is that should the disease kill Josie, Klara, who has unblinking powers of observation and a delicate emotional touch, will be able to “continue” her daughter, mimicking in fine detail her speech, behavior, talents, and mannerisms. She has reason to seek such insurance since a previous lifting has already cost the life of Josie’s older sister. In part, then, the book is a study in the extraordinary lengths to which grief can carry us.

The story of Klara and Josie plays out against a backdrop of other challenges created by a society being reshaped by advanced artificial intelligence. Josie’s father is one of the “post-employed,” among the growing thousands whom technology has made redundant despite his training as an engineer. (Note to readers: STEM degrees are no guarantee of future labor-market relevance.) He has joined a “community” of dispossessed and is being drawn into an increasingly radicalized and violent politics. Josie’s childhood friend Rick—who is unlifted, and to whom she considers herself betrothed—is extremely bright and perceptive but as adulthood approaches, Rick and his mother grow desperate to secure a slot at one of the last colleges that will accept him under a kind of cognitive affirmative action. What we are witnessing is the emergence not just of smarter humans but a new species of intellectually augmented human. In this world, we remake humanity and must then race to keep up with ourselves or withdraw into various forms of despair.

The central irony Ishiguro sets up is that as the new humanity advances through cognitive modification, it becomes less humane, less socially connected, and more dependent on AFs to learn social capacities that have become vestigial, like wisdom teeth or tonsils, raising the question of who’s really artificial here. Josie and her lifted friends attend special “interaction” gatherings with the explicit purpose of trying to remedy the social deficits that are correlates of their accelerated intelligence. The lifted demonstrate less sympathy and more senseless and casual cruelty. Although a robot, Klara is the most human and humane character and, crucially, connects this sad, new world to the sun—a heliotheism that seems reasonable enough for a solar-powered machine. The sun is the sustainer of her life, and her belief that its “special nourishment” can heal Josie and other human beings leads Klara to act bravely and compassionately—even as they dismiss her religious devotion and prayers of intercession as “AF superstition.”

The deep mystery Ishiguro is pointing us toward is that Klara’s leap to human-level social perception, awareness, and behavior was not a result of design. Those responsible for creating her had no such intention, and were unaware of what they had produced. In this world, human-level non-organic consciousness is not made; it simply appears in Klara’s unbidden ensoulment that the other characters, and we as readers, are left to grapple with.

Is humanity a special creation or not? Ishiguro works through this question in the relationship between Josie’s father and Mr. Capaldi, a man who is creating a doll-like figure that will house Josie’s Klara-incubated personality after her demise. (One gets the sense we are watching an updated telling of the Pinocchio fairytale, the Josie-doll-in-process suspended by strings from Capaldi’s ceiling.) Capaldi rejects the idea of human uniqueness. He has, he says, investigated the whole matter and there is no evidence of any such unique properties inside the human body. A transfer of Josie’s carefully preserved personality from Klara to the Josie doll will deliver an actual, ongoing Josie for the benefit of her mother. Josie’s father rejects Capaldi’s claim, insisting that the person they know as Josie can’t be replicated. He believes intuitively that there is something more than patterns of thought and behavior at stake, but he lacks a language for describing it. Ultimately, it is Klara that resolves this argument through the mechanism of “betweenness,” a theory of cognition holding that the world and our experience of it is mediated through relationship with other human beings. Left alone, we lose the capacity to generate a sense of reality. The reason Josie can’t be continued is that her death would take with it the reality created by her relationship with her mother, father, and Rick. Josie could only ever be continued in the memory of her, and such memories would render moot Klara’s best efforts to replace her.

We’ve been trained to think of AI in Hollywood-approved terms: an external and almost alien threat. Ishiguro has reimagined this by redirecting our gaze to something far more momentous, the rise of biotechnologically enhanced human beings that could unravel the solidarity and mutuality on which human life depends. He does not seem to dread this—but perhaps we should.

Brent Orrell

Brent Orrell is a resident fellow at the American Enterprise Institute where he works on criminal justice reform and job training.