Using AI, cartoonist Amy Kurzweil connects with deceased grandfather in 'Artificial'

When it comes to attempts by the living to communicate with the dead, psychic mediums and ouija boards will probably never go out of style. But now, there's also AI.

In her new graphic memoir, Artificial: A Love Story, New Yorker cartoonist Amy Kurzweil describes how she and her father — famed futurist, technologist and inventor Ray Kurzweil — harnessed the power of artificial intelligence to connect with the grandfather she never knew.

Fredric Kurzweil died in 1970, more than a decade before his granddaughter Amy was born. But Amy said Fred has always had this epic presence in her family.

"The story I grew up hearing about my grandfather was essentially that music saved his life," Amy told NPR in an interview in San Francisco.

Fred's journey is at the heart of Artificial: A love Story.

The author said that in 1937, when her grandfather — a renowned Austrian conductor, pianist and music educator — was in his 20s, the American philanthropist Gertrude Sumner Ely heard him perform a concert in Vienna.

"And she was so impressed with his talent that she said to him, 'If you ever need anything, let me know,'" Amy said.

The following year, the Nazis annexed Austria. Amy said her grandfather, who was Jewish, needed to escape.

So Fred wrote to Ely. The philanthropist subsequently sponsored his U.S. visa.

"This is the story I grew up with about him, is that really his artistry, his talent is what enabled him to get out of Vienna," Amy said.

Creating Fredbot

Amy's father Ray Kurzweil said he was 22 years old when his father died.

Ray is known as the inventor of an early reading machine for the blind and a revolutionary line of electronic keyboards.

He's also a futurist, who has been predicting the merging of humans and machines for decades in his books, like The Singularity is Near from 2005 and his forthcoming title, The Singularity is Nearer.

"To have large language models appear this year is about right on track," Kurzweil said, in reference to the algorithms used to train AI tools like GPT and DALL-E that have stormed the headlines lately. "It's going to be fantastic progress in the next few years."

In 2016, as Amy chronicles in her book, her father embarked on a project using AI to bring his father Fred back to life — sort of.

"I missed him," Ray said. "So I created a technology that seemed like talking to him. You could ask him anything, and he would actually answer something similar to what I remember he would have said."

Building a chatbot version of his father (also known as Fredbot) involved digitizing thousands of Fred Kurzweil's documents, including lecture notes, love letters, musical writings, and letters seeking employment.

Amy said the family storage unit was buckling under the weight of her grandfather's papers.

"We just have boxes and boxes and boxes of documents," she said.

So she offered her assistance. Amy said trawling through Fred's voluminous archives, transcribing the findings, and then documenting many of these materials in her book, helped her to understand her grandfather better.

"In the case of Viennese and other types of Jews, documentation was how you saved your life," Amy said. "Through waiting in lines to show somebody a paper that proved that they didn't have a dog, or they had these things in their bank account that they were going to forfeit, that was your ticket to leave this bureaucratic fascist regime."

Advantages and limitations

Fredbot is a text-based program. Amy and Ray haven't been able to track down recordings of Fred Kurzweil's voice, though recordings of some of his musical performances by the likes of Chopin, Beethoven and his favorite composer Brahms are available online.

So to interact with the bot (which isn't publicly available, by the way) you type questions into a Web-based interface, and the system responds with relevant lines culled from the archives. In other words, Fredbot's responses are things Fred Kurzeil actually wrote. Amy includes some of these interactions in her book, for example:

AMY: What do you love most about music?
FREDBOT: Melody, the element which makes the most direct appeal, is what is generally remembered – whistled and hummed.

Amy said her grandfather was quite shy. It was through his music that he fully expressed himself.

"When he was performing his extroverted self would come out," she said.

But Fredbot doesn't feature any of Fred's music. Acknowledging the limitations of the bot, Amy said this might be possible in the future.

"When I started this book seven years ago, the concept of a chatbot built from archival documents seemed very sci-fi," she said. "And now the technology is outdated."

But she said she still gets a lot out of interacting with Fredbot.

"It's not exactly like having a conversation with him in person, but it's a reflection of him," she said.

Technology and humanity

More recent developments in the reanimate-the-dead chatbot space do feature the voices of the deceased, as is the case with the offering from HereAfterAI.

"It's really just in the last year or so with generative models that suddenly we have the ability to have much more fluid, much more human-like chatbots," said James Vlahos, founder of HereAfter AI and a conversational AI specialist with the social robotics company Embodied. "Now we can have much more intelligent conversations than we ever had before."

Amy Kurzweil said the chatbot project and the book that came out of it underscored her somewhat positive feelings about AI.

"I feel like you need to imagine the robot you want to see in the world," she said. "We're not going to stop progress. But we can think about applications of AI that facilitate human connection."

Fellow New Yorker cartoonist Kristen Radtke said given the Kurzweil lineage, it's no surprise Amy is overall optimistic about the future of AI.

"It's funny to see AI suddenly be like the hot button issue that everyone is talking about," Radtke said. "And Amy has really been engaging with these ideas her entire creative life."

But at heart, Radtke said, the superpower of Artificial: A Love Story is its grasp of what, amid all of these technological advances, it means to be human.

"Amy approaches everything so philosophically and so emotionally and kind of spiritually," Radtke said.

One way in which these qualities show up in the book is in the author's attempts, over and over again, to duplicate materials from her grandfather's archives through her meticulous drawings.

At first glance, the repeated images appear identical. But look more closely, and they all have these tiny variations. Like memories, drawings cannot be recreated in exactly the same way each time.

Radtke said with this approach, the author is making the point that no matter how hard artists try, they will always lack the precision of a robot: "Showing the fact that there's a human hand behind these drawings."

Audio and digital stories edited by Meghan Collins Sullivan. Audio produced by Isabella Gomez-Sarmiento.