Can AI Truly Give Us a Glimpse of Lost Masterpieces?
IN 1945, FIRE claimed three of Gustav Klimt's most controversial paintings. Commissioned in 1894 for the University of Vienna, "the Faculty Paintings"—as they became known—were unlike any of the Austrian symbolist's previous work. As soon as he presented them, critics were in an uproar over their dramatic departure from the aesthetics of the time. Professors at the university rejected them immediately, and Klimt withdrew from the project. Soon thereafter, the works found their way into other collections. During World War II, they were placed in a castle north of Vienna for safekeeping, but the castle burned down, and the paintings presumably went with it. All that remains today are some black-and-white photographs and writings from the time. Yet I am staring right at them.
Well, not the paintings themselves. Franz Smola, a Klimt expert, and Emil Wallner, a machine learning researcher, spent six months combining their expertise to revive Klimt’s lost work. It’s been a laborious process, one that started with those black-and-white photos and then incorporated artificial intelligence and scores of intel about the painter’s art, in an attempt to recreate what those lost paintings might have looked like. The results are what Smola and Wallner are showing me—and even they are taken aback by the captivating technicolor images the AI produced.
Smola and Wallner find this delightful, but not everyone supports AI filling in these voids. The idea of machine learning recreating lost or destroyed works is, like the Faculty Paintings themselves, controversial. “My chief concern is about the ethical dimension of using machine learning within the context of conservation,” says art conservator Ben Fino-Radin, “because of just the sheer volume of ethical and moral issues that have plagued the machine learning field.”
To be sure, the use of technology to revitalize works of human artistry comes riddled with thorny questions. Even if there was a perfect AI that could figure out what colors or brushstrokes Klimt might have used, no algorithm can generate authorial intent. Debates about this have been raging for centuries. Back in 1936, before Klimt’s paintings were destroyed, essayist Walter Benjamin argued against mechanical replication, even in photographs, saying that “even the most perfect reproduction of a work of art is lacking in one element: its presence in time and space, its unique existence at the place where it happens to be.” This, Benjamin wrote in The Work of Art in the Age of Mechanical Reproduction, is what he called a work’s “aura.” For many art lovers, the notion of a computer reproducing that intangible element is preposterous, if not downright impossible.
To do this, Wallner developed and trained a three-part algorithm. First, the algorithm was fed some hundred thousand images of art from the Google Arts and Culture database. This helped it understand objects, artwork, and composition. Next, it was schooled specifically in Klimt’s paintings. “This creates a bias toward his colors and his motifs during the time period,” Wallner explains. And finally, the AI was fed color clues to specific parts of the paintings. But with no color references to the paintings, where did these clues come from? Even Klimt expert Smola was surprised by how much detail the writings of the time revealed. Because the paintings had been considered so sordid and weird, critics tended to describe them at length, right down to the artist’s color choices, he says. “You can call it an irony of history,” says Simon Rein, the project’s program manager. “The fact that the paintings caused a scandal and were rejected puts us in a better position to restore them because there was so much documentation. And those kinds of data points, if fed into the algorithm, create a more accurate version of how these paintings probably looked at the time.”
“As soon as you see a black-and-white picture, the first thing you do is imagine what it would look like: You assume things about a painting; you see the sky as blue,” Wallner says. As he watched the image generate, a swirling, mysterious green-tinted sky appeared in the rendering on his screen. “That was the shocking part because you see your bias,” he says. “For me, the first moment when I saw these paintings in color was like wow, this is what it looks like!”
Post a Comment