The Unsettling Performance That Showed the World Through AI’s Eyes

Artist Trevor Paglen is best known for images of the security state. Now he's collaborating with Kronos Quartet and Obscura Digital to explore AI.

Inside an abandoned warehouse on the San Francisco docks, as the damp air floods through the holes in its rusted tin roof, Sunny Yang is playing her cello while recovering from the flu. She is 45 percent sad and 0.01 percent disgusted.

That, at least, is the read from the AI that's tracking her expressions, gestures, and body language from the other side of the warehouse, flashing these stats on the movie screen behind her. The audience---several hundred people huddled between her and the AI, dressed in scarfs, hats, and overcoats---lets out a collective laugh.

Yang is playing alongside the rest of the Kronos Quartet, the iconic San Francisco string ensemble known for its unorthodox experimentation, and the AI is obeying orders from Trevor Paglen, the American artist who poses big questions about technology and surveillance through nearly any medium he can get his hands on. It's all part of Sight Machine, a Paglen-orchestrated performance that explores the rise of computer vision.

Minutes later, as the quartet begins another piece, new images appear on the screen. At first, they show the Earth through the eyes of a satellite circling above. Then they zoom in on the ground below, an AI locks in on homes, cars, and individuals, tracking their movements from the heavens much as Paglen's hardware and software tracked Yang's movements inside the warehouse. "One Earth, one people," says a disembodied voice, the words bouncing through the cold of the warehouse. This time, no one laughs. What was amusing just minutes earlier is now so unsettling.

Three months after Paglen's piece at the edge of San Francisco Bay, these feelings still resonate. As is typical of the artist's work, Sight Machine tapped into something that's largely unseen but very real. As computer vision quietly spreads through our lives and landscapes, it's entertaining and practical, powerful and flawed, amusing and disturbing. The same goes for AI as a whole. You can't see it. But it's everywhere.

"There were no conclusions," said Henry Dills, a photographer and cellist who watched the performance dressed in a brown sport coat and a white scarf that reached past his waist. "These machines are starting to massively overshadow us. It used to be God. Now it's machines."

On the AI Train

Google, Facebook, and Apple are all building services that can analyze human emotion in real time. Startups like Descartes Labs and Orbital Insights use similar technology to analyze vast troves of satellite imagery in order to understand human activity and intentions that humans themselves would have trouble drawing on their own. Relying on deep neural networks---complex mathematical systems that can learn to perform tasks by analyzing vast amounts of data---these services don't work perfectly. But they're improving rapidly---and they're rapidly moving from the lab into the real world.

That includes Paglen's technology, which he and his team of engineers built using open source software that runs neural networks inside Google and other companies. Some of the imagery during the concert came pre-recorded, but in many cases, neural nets tracked Yang and the rest of the Kronos quartet in real-time thanks to collaboration with light projection company Obscura Digital. "What I want out of art are things that help us see the world that's around us, the historical moment we live in," Paglen says.

Paglen is best known for work that explores the depths of government surveillance, from photos of undersea cables tapped by the NSA to a book that maps the Pentagon's global spy network. This year, he began a residency at Stanford's Cantor Arts Center, next to a major hub of artificial intelligence research.

But he isn't toying with old ideas or cultural cliches about robot overlords. He's exploring what's happening at the sharp end of computer vision. Originally, Paglen wanted a residency inside OpenAI, the billion-dollar lab bootstrapped by Tesla CEO Elon Musk and Y Combinator president Sam Altman. OpenAI aims to accelerate the progress of artificial intelligence even as it protects the world from the dangers such acceleration may bring. But OpenAI balked. So Paglen went to Stanford.

His new residency touches on some of the same themes that run through his work on government surveillance. The two overlapped when the quartet played Terry Riley's One Earth, One People, One Love and those AI overlords zoomed in on us from above (and the dreadlocked Silicon Valley security and privacy guru Moxie Marlinspike watched from his spot at the very center of the warehouse.) At the same time, Paglen is addressing the sometimes mysterious, sometimes unsettling way that modern AI learns on its own.

In many cases, neural networks are enormously capable at doing what they're asked to do. But even the people who build them don't completely understand why they're so effective. They learn by analyzing more data, more carefully than a human ever could. This complexity means, among other things, that humans can't really dissect decisions they make.

Sight Machine, Paglen says, aims to explore this mystery. "It's trying to look inside the software that is running an AI. It's trying to look into the architectures of different computer vision systems and trying to learn what it is that they are seeing," he explains. "How are they looking at images? And what are the social, ethical, economic, and political consequences of these modes of seeing, which are becoming more and more ubiquitous?"

This came to the fore at the end of the performance, when Kronos played the first movement of Steve Reich's Different Trains, what Paglen calls a musical exploration of train lines driving the expansion of the United States in the early part of the 20th century. It was a play on words. "I like the idea of trains and training sets," Paglen says, letting out one of his big staccato laughs. But it also served as a symbol for the way neural networks are careening into the future, largely without our help.

As Kronos played, countless photos flashed on the screen, all lifted from ImageNet, one of the chief image databases for training computers to recognize specific images. It was beautiful, almost mesmerizing---like so much of today's AI research. But Paglen was also hinting that this beauty may morph into something else---echoing the worry of so many that AI will not just destroy our privacy but steal our jobs and perhaps even grab control over our own world. Like Reich, Paglen is exploring the relationship between technology and progress, as he wrote in the program for Sight Machine. "Progress" was in quotes.