Director Paul Trillo’s new music video for the band Washed Out is the first commercial use of OpenAI’s new text-to-video generator, Sora. “The Hardest Part” tells the story of a couple’s life together as an infinite dolly shot, from meet-cute in school through adulthood.
“This was an idea I had almost 10 years ago and then abandoned,” says Trillo. “With Sora, I was finally able to bring it to life.”
The filmmaker, one of the first to be granted access to OpenAI’s new text-to-video generator, says it took 700 generations to create the 55 selected shots that make up the video.
“I really wanted to do something that was both unique to the hallucinations of Sora while also attempting to create something that felt timeless and works regardless of what technology was being used,” explains Trillo in an interview.
Years ago, he “wanted to do a continuous zoom through a relationship that spans multiple decades beginning in the late ’70s,” but working with a dozen actors through several time periods, not to mention “a whole slew of locations,” he says, wasn’t realistic for the music video budgets he works with.
But, Trillo explains, Sora presents unique challenges and opportunities to artists. “I think fundamentally, what works as a story, what works on an audience will never change. But our process to get to that finish line is different,” he says.
“What is unique about AI is that it’s this more fluid, organic process where you’re ideating. You have your idea, it feeds into the final product, and then the final product gives you a new idea to go back into the writing phase and rewrite,” Trillo adds. “In this way, pre-production, post-production become really fluid. That’s what’s been really exciting for me: you can actually spend more time trying to craft that story.”
Ernest Greene, the artist behind Washed Out, says he’s been a fan of Trillo’s for a long time. “He was at the top of my list of potential collaborators,” Greene says. “What Paul has come up with is nostalgic, sad, uplifting and often quite strange. However, he still manages to make you feel for the characters and invested in the journey of how their lives progress.”
Trillo’s work is characterized by a visionary use of technique and technology in service of story. “The Hardest Part” marks Trillo’s 20th Vimeo Staff Pick. His other award-winning short-form work includes “At the End of the Cul-de-sac,” a 10-minute single-take shot with a drone, the first short film made with Runway AI Gen-2 “Thank You For Not Answering,” and a collaboration with French artist Jacques titled “Absolve,” which was shot on location in the Louvre.
In addition to Trillo’s narrative and video artwork, he has also directed music videos for The Shins and ads for Ford, Toyota, IBM, Microsoft, Starbucks, T-Mobile, and a Super Bowl commercial for Carvana, which have established him as one of the most eclectic, creative, innovative directors working on the cutting edge of technology today.
“Something I noticed that Sora does really well is making organic transitions between one scene that bleeds into another. And the walls don’t necessarily make sense for how one environment, you know, ends and another one begins,” Trillo says. This never-ending zoom technique is something he has used in his work for over 10 years. Trillo immediately thought of how he could use Sora to recreate it. Once he narrowed in on that, even some of the people on the Sora team were surprised.
“When I had started to do these really fast, like, hyperlapse things, they’re like, ‘Oh, we didn’t even know it could do that.’ The prompt becomes this chemistry formula of words sequenced, timed and paced in a very specific order,” he explains. “You can trigger events to happen. You can say, like this, then this. So you’re almost just writing a script. Some of these prompts are very detailed.”
You can check out the incredible script-like prompt Trillo gave Sora to produce the infinite dolly shot that drives the video here. “The right combo of words created that hyperlapse, the infinite dolly shot,” he says. “And once I kind of figured that equation out, I would just replace the subjects and the location and the events that are happening.”
At the beginning of April, two weeks before the prestigious TED conference in Vancouver, Jonathan Wells, who commissions video for TED, contacted Trillo about creating something that envisions what the next 40 years of TED would look like. This was a tremendous opportunity for a visionary filmmaker to get his work in front of some of the world’s most influential people.
In order to convey this never-ending road of ideas and technological progress, Trillo used the hyperlapse technique he developed making “The Hardest Part,” prompting Sora to fly through future iterations of the TED conference.
“It’s an indication, I guess, of this new kind of content that is wildly ambitious and weird and has a somewhat acceptable production value that can be turned around in a short amount of time,” Trillo said.
Charlie Fink is the author of the AR-enabled books “Metaverse,” (2017) and “Convergence” (2019). In the early 90s, Fink was EVP & COO of VR pioneer Virtual World Entertainment. He teaches at Chapman University in Orange, CA. Follow him on Twitter or LinkedIn, and check out his website or other work.
More from AR Insider…
