Runway AI's Gen-4: How Can AI Montage Go Beyond Absurdity
NEW YORK, NEW YORK - MAY 09: (L-R) Jane Rosenthal and Cristobal Valenzuela speak onstage during the ... More 2024 AI Film Festival New York Panel at Metrograph on May 09, 2024 in New York City. (Photo by)
The recent release of Runway AI's Gen-4 has ignited both excitement and existential questions about the future of cinema and media in general. With the company now valued at 3 billion following a $308 million funding round led by the private equity firm General Atlantic and backed by industry heavyweights like Nvidia and SoftBank, AI's march into Hollywood appears unstoppable.
The film industry, alongside all creative sectors, from digital marketing to social media, stands at a technological crossroad. As artificial intelligence begins to reshape every aspect of visual storytelling and change the landscape of entertainment and digital commerce, we must assess its potentials and pitfalls.
Major production companies are rapidly adopting AI video tools. Fabula, the acclaimed studio behind Oscar-winning A Fantastic Woman and biopic Spencer, just announced a partnership with Runway AI to integrate AI across its production pipeline. Lionsgate signed a deal with Runway last fall to explore AI-powered filmmaking. Experimental directors like Harmony Korine have already debuted AI-assisted film at Venice last year.
The broad applications of AI videos are already impressive, from pre-visualizing scenes for Amazon's House of David to creating advertisement for Puma. Yet beneath these flashy demonstrations lies a more fundamental question: can AI-generated content evolve beyond technical spectacle to deliver truly meaningful stories?
Runway's Gen-4 represents significant progress in several areas: character consistency, scene coherence, and visual fidelity. An example Runway AI releases show two main characters stay consistent across different shots ranging from walking, running, petting a cow, lighting up a match, and maintain fidelity of the look of a steppe in gloomy weather.
Yet these technical improvements don't address the core challenge: AI excels at generating individual moments but struggles with coherent and sustained storytelling. While it can create a stunning shot of giraffes and lions roam in the New York City, can it make audiences care about a city turned into a zoo?
AI videos risk repeating the early mistakes of Computer Generated Imagery (CGI), prioritizing visual gimmicks over in-depth messages. As barriers to creative production and film making disappear, we may face a flood of visually polished but emotionally hollow contents, derivative works optimized for algorithmic efficiency, or compelling synthetic media that lacks human touch. While AI videos can wow first-time viewers, can they make audiences want to watch them more? Can AI films ever produce classic pieces that draw generations of movie-goers?
Current multi modal AI technologies center on innovations in film, media, and video games. A recent project spearheaded by researchers from Nvidia, Stanford and UCSD uses Test-Time-Training layers in machine learning models to generate 60-seconds animations of Tom and Jerry. To achieve this, the team trained the model on 81 cartoon footages between 1940 and 1948, which add up to about 7 hours. The model generates and connects multiple 3-second segments, each guided by storyboard annotating plots, settings, and camera movements. The technique highlights significant potential to scale video productions and animation series creations.
A poster for Joseph Barbera and William Hanna's 1950 cartoon 'The Framed Cat'. (Photo by Movie ... More Poster)
But the technology also reveal critical flaws that persist among AI video generators such as Sora, Kling, Runway, Pika, etc. One limitation is continuity error. For example, rooms, landscapes, and lighting shift unnaturally between 3-second segments. Physics defiance is another problem. For instance, in the earlier mentioned Tom and Jerry AI videos, Jerry's cheese float or morph into different sizes and textures at segment boundaries. Another issue is narrative disjointedness. As the segmentation of content is necessary for algorithms to effectively learn the contents, understand the prompt, and accurately generate videos, AI models struggle to show logical scene progression.
These traces of what I call AI montage also appear in Runway AI's videos, the elephants walking across the Time Square is abruptly followed by a scene of a cheetah running across a bridge. One is set in cloudy weather while the next in a sunny day. The changes do not push the storyline forward nor do they convey any logic. The absurd, the fragmented, and the incongruous, are what AI video generators currently good at producing. For now, AI struggles to replicate the coherence of even a 5-minute cartoon, let alone a feature film.
AI-generated videos show strength as a medium for critiquing both itself and the societies that produce it. Director Jia Zhangke's recent AI film made using Kling AI imagines a future run by robotic caretakers. The film provokes audiences to reflect on the crisis of aging populations, societal neglect, and the erosion of empathy in an era of breakneck competition, capitalism, and exploding automated technologies.
Jia's film show robot companions taking the elderly for a walk or helping them harvest crops, in lieu of real sons and daughters. Such a theme is grounded in societal challenges today. The film critiques the substitution of human connection with automated machines and transactional relationships, and raises the concern over relentless stress and long hours in workplaces.
Just as Charlie Chaplin used industrialization-era tools to critique industrialization in Modern Times, today's filmmakers can use AI to critique the conditions of its own existence. Consider how synthetic news anchors might expose media manipulation, or endlessly combinable streaming content could comment on algorithmic culture.
Just like science fictions that critique environmental disasters, human greed, and inequality, the most compelling AI films will likely be those that embrace their own artificiality to engage with real social problems.
Rather than fearing obsolescence, filmmakers might focus more intensely on what machines cannot replicate: the nuance of human emotions, complexities of human nature, the weight of lived experience, and the cultural resonance of authentic storytelling.
History suggests that film and media have always adapted to technological upheaval, from silent to sound, black-and-white to color, celluloid to digital, each time emerging with new creative possibilities. The question is no longer whether AI will change filmmaking, but how filmmakers will harness it to tell stories that matter.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Los Angeles Times
4 hours ago
- Los Angeles Times
Journalists get a guided tour of totalitarianism in ‘Meeting with Pol Pot'
French Cambodian director Rithy Panh has often cited the genocidal regime of the Khmer Rouge, which killed his family and from which he escaped, as the reason he's a filmmaker. His movies aren't always directly about that wretched time. But when they are — as is his most memorable achievement, the Oscar-nominated 2013 documentary 'The Missing Picture,' which re-imagined personal memories using clay-figurine dioramas — one senses a grand mosaic being assembled piece by piece linking devastation, aftermath and remembrance, never to be finished, only further detailed. His latest is the coolly observed and tense historical drama 'Meeting With Pol Pot,' which premiered last year at Cannes. It isn't autobiographical, save its fictionalization of a true story that happened concurrent to his childhood trauma: the Khmer Rouge inviting a trio of Western journalists to witness their proclaimed agrarian utopia and interview the mysterious leader referred to by his people as 'Brother No. 1.' Yet even this political junket, which took place in 1978, couldn't hide a cruel, violent truth from its guests, the unfolding of which Panh is as adept at depicting from the viewpoint of an increasingly horrified visitor as from that of a long-scarred victim. The movie stars Irène Jacob, whose intrepid French reporter Lise — a perfect role for her captivating intelligence — is modeled after the American journalist Elizabeth Becker who was on that trip, and whose later book about Cambodia and her experience, 'When the War Was Over,' inspired the screenplay credited to Panh and Pierre Erwan Guillaume. Lise is joined by an ideologically motivated Maoist professor named Alain (Grégoire Colin), quick to enthusiastically namedrop some of their hosts as former school chums in France when they were wannabe revolutionaries. (The character of Alain is based on British academic Malcolm Caldwell, an invitee alongside Becker.) Also there is eagle-eyed photojournalist Paul (Cyril Gueï), who shares Lise's healthy skepticism and a desire to learn what's really happening, especially regarding rumors of disappeared intellectuals. With sound, pacing and images, Panh readily establishes a mood of charged, contingent hospitality, a veneer that seems ready to crack: from the unsettlingly calm opening visual of this tiny French delegation waiting alone on an empty sun-hot tarmac to the strange, authoritarian formality in everything that's said and shown to them via their guide Sung (Bunhok Lim). Life is being scripted for their microphones and cameras and flanked by armed, blank-faced teenagers. The movie's square-framed cinematography, too, reminiscent of a staged newsreel, is another subtle touch — one imagines Panh rejecting widescreen as only feeding this evil regime's view of its own righteous grandiosity. Only Alain seems eager to ignore the disinformation and embrace this Potemkin village as the real deal (except when his eyes show a gathering concern). But the more Lise questions the pretense of a happily remade society, the nervier everything gets. And when Paul manages to elude his overseers and explore the surrounding area — spurring a frantic search, the menacing tenor of which raises Lise's hackles — the movie effectively becomes a prison drama, with the trio's eventual interviewee depicted as a shadowy warden who can decide their fate. Journalism has never been more under threat than right now and 'Meeting with Pol Pot' is a potent reminder of the profession's value — and inherent dangers — when it confronts and exposes facades. But this eerily elegiac film also reflects its director's soulful sensibility regarding the mass tragedy that drives his aesthetic temperament, never more so than when he re-deploys his beloved hand-crafted clay figurines for key moments of witnessed atrocity, or threads in archival footage, as if to maintain necessary intimacy between rendering and reality. Power shields its misdeeds with propaganda, but Panh sees such murderous lies clearly, giving them an honest staging, thick with echoes.
4 hours ago
R. Kelly claims prison officials plotted to kill him. Judge denies his release
SPRINGFIELD, Ill. -- A judge has denied singer R. Kelly 's request to be freed from prison, saying she lacks jurisdiction to consider the convicted sex offender's allegations that federal prison officials plotted to murder him. U.S. District Judge Martha M. Pacold canceled Friday's scheduled arguments in Chicago, and instead issued a five-page ruling denying his emergency request to serve his time on home detention during a temporary furlough. Born Robert Sylvester Kelly, the 58-year-old Grammy-winning R&B singer is serving sentences at a prison in Butner, North Carolina, for child sex crimes and racketeering. Pacold noted that federal courts have only limited power in such cases, and one by one she examined and rejected various ways Kelly's request could be considered. 'Jurisdictional limitations must be respected even where, as here, a litigant claims that the circumstances are extraordinary,' Pacold wrote. "Kelly has not identified (nor is the court aware of) any statute or rule that authorizes the court to exercise jurisdiction. Thus, the court cannot award relief in this case.' Kelly claimed in a series of filings this month that prison officials solicited white supremacist leaders to kill him, to halt the disclosure of information damaging to prison officials. Kelly's attorney says he was then moved to solitary confinement and purposely given an overdose of medication, which required hospitalization and surgery for blood clots. Pacold noted that Kelly offers no evidence to back up his claims. Government lawyers suggested Kelly had spun a 'fanciful conspiracy.' Known for such hits as 'I Believe I Can Fly,' Kelly was found guilty in Chicago in 2022 of three charges of producing child sexual abuse images and three charges of enticement of minors for sex. In New York in 2021, he was found guilty of racketeering and sex trafficking. He is serving most of his 20-year Chicago sentence and 30-year New York sentence simultaneously. His appeals have been unsuccessful, including to the U.S. Supreme Court. Kelly has also sought President Donald Trump's help.
USA Today
4 hours ago
- USA Today
Tom Brady embodies inner-heel, rips up Eli Manning jersey
Who would have ever thought legendary quarterback Tom Brady and World Wrestling Entertainment champion John Cena would both go heel (turn into a bad guy) in the same year? Brady made a grand entrance at Fanatics Fest in New York City on Friday. The event had a mini WWE entrance set, and the seven-time Super Bowl-winning quarterback entered to former WWE champion Cody Rhodes' theme. Brady came onstage and posed with the title on his shoulder. He then slowly made his way down the ramp, stopping to snatch a Sauce Gardner jersey from someone in the crowd, slamming it on the ground and spitting on it. As he made his way further down the ramp, he was handed an Eli Manning jersey. He held the jersey up for everyone to see and then ripped it in half. "Let's go," Brady yelled out to the crowd. Manning then came out of nowhere, jumped on Brady's back and rode him back up the ramp. It served as the perfect entertainment to kick-start the three-day event. Fanatics Fest NYC is a massive gathering featuring sports and collectibles. Fans from all over the world travel to the event to buy, sell and trade. They also have an opportunity to meet their favorite athletes and get exclusive merchandise. Brady seemed to be in his element playing the villain role in the world of professional wrestling. His former teammate, Rob Gronkowski, had a Wrestlemania moment back in April 2017. Perhaps it's time for Brady to do the same. Follow Patriots Wire on Twitter and Facebook.
