When you feed nothing, what happens?
Terence Broad: From Artificial Intelligence to (Un)stable Equilibrium: A Journey through the Nature of AI and GAN-Generated Art
Broad did not train his artificial intelligence on any data at all. By hacking a neural network, and locking elements of it into a recursive loop, he was able to induce this AI into producing images without any training data at all — no inputs, no influences. Depending on your perspective, Broad’s art is either a pioneering display of pure artificial creativity, a look into the very soul of AI, or a clever but meaningless electronic by-product, closer to guitar feedback than music. The way to use generative AI beyond the large-scale manufacture of derivative slop is a suggestion by his work.
Talking to him about his process, and reading through his PhD thesis, one of the takeaways is that, even at the highest academic level, people don’t really understand exactly how generative AI works. Midjourney has an exclusive emphasis on “prompt engineering” and allows users to adjust a nearly endless number of settings and elements. We know that if we feed generative AI data, a composite of those inputs will come out the other side, but no one really knows, on a granular level, what’s happening inside the black box. (Some of this is intentional; Broad notes the irony of a company called OpenAI being highly secretive about its models and inputs.)
If you stumbled across Terence Broad’s AI-generated artwork (un)stable equilibrium on YouTube, you might assume he’d trained a model on the works of the painter Mark Rothko — the earlier, lighter pieces, before his vision became darker and suffused with doom. The images are made from fields of pure color, but they are morphing into other forms and colors.
In 2018, Broad started a PhD in computer science at Goldsmiths, University of London. It was there, he says, that he started grappling with the full implications of his vow of data abstinence. “How could you train a generative AI model without imitating data? It took me a while to realize that that was an oxymoron. A generative model is a statistical model of data that mimics the data it has been trained on. I had to find other ways of framing the question. Broad started paying attention to the GAN, an artificial intelligence model, which was quite popular at the time. Two neural networks combine to train each other. Both networks analyze a dataset, and then the generator attempts to fool the discriminator by generating fake data; when it fails, it adjusts its parameters, and when it succeeds, the discriminator adjusts. The tug-of-war between discriminator and generator at the end of this training will produce an ideal equilibrium that will enable this GAN to produce data that is on par with the original training set.
Legal threats from a multinational corporation pushed him further away from inputs. One of Broad’s early artistic successes involved training a type of artificial neural network called an autoencoder on every frame of the film Blade Runner (1982), and then asking it to generate a copy of the film. The result, bits of which are still available online, are simultaneously a demonstration of the limitations, circa 2016, of generative AI, and a wry commentary on the perils of human-created intelligence. The video was posted online and received major attention from both Broad and Warner Bros., as well as a notice from the Digital Millennium Copyright Act. “Whenever you get a DMCA takedown, you can contest it,” Broad says. It was not something I was willing to risk as a graduate with lots of debt because you made yourself liable to be sued in an American court.
When McNitt cut to quick shots of different animals nurturing their young and close-ups of holes being filled in by organisms, you can tell that the visuals were a big part of the film. They each feel like another example of text-to-video models’ ability to churn out uncanny-looking, decontextualized footage that would be difficult to incorporate into fully produced film. But in the behind-the-scenes making-of video that Google shared in its announcement last week, McNitt speaks at length about how, when faced with the difficult prospect of having to cast a real baby, it made much more sense to her to create a fake one with Google’s models.
Producing films with infants poses a lot of production challenges and that’s not an issue with dolls and babies. But going the gen AI route also presented McNitt with the opportunity to make her film even more personal by using old photos of herself as a newborn to serve as the basis for the fake baby’s face.
It is very sentimental, but the message of the power of a mother’s love is cliched when it is compared with nature footage. Ancestra is a project that is trying to prove that all the videos on the internet are actually exciting. The film has a lack of interesting narrative substance that makes it seem like a weak argument for Hollywood to get to the slop trough while it’s hot.
The biggest revelation of the behind the-scenes video is how small the production team was for Ancestra, compared to what would be seen on a traditional short film. The film would have been more expensive and time consuming if more artists had been hired to work on the film. Especially for indie filmmakers and up-and-coming creatives who don’t have unlimited resources at their disposal, those are the sorts of challenges that can be exceedingly difficult to overcome.
The technology used to make films with generative artificial intelligence is not good enough to make art that will push people to sign up for another streaming service, for example, or to make movies that will piss people off. And it’s important to bear in mind that, at the end of the day, Ancestra is really just an ad meant to drum up hype for Google, which is something none of us should be rushing to do.
