There seems to be a new AI video announcement every day and the latest is from Hedra, a startup taking a character-first approach to putting ideas into motion.
This week alone we've seen new features for Luma Labs Dream Machine and the announcement of Runway's new Sora-like Gen-3.
Character-1 is a research preview of their upcoming foundation video model that gives users fine-grained control over how virtual characters are animated using AI.
In the preview, you can give it audio, and a picture and watch as it creates a lip-synced video showing that character in your picture talking. Unlike other lip-sync tools this adds greater levels of expression and movement than I’ve seen before.
Hedra is free during this research preview and you can create any length of video. The company are using this to test issues with both the model and their moderation tools before rolling out more advanced features.
How does Hedra Character-1 work?
Character-1 is a new foundation AI model designed to create fully controllable and realistic characters using AI. The company says they'll be able to do expressive talking, singing and even rapping with potentially infinite durations.
Right now using it is fairly simple. Once registered you either generate audio from text or give it your own audio and create a character. This can be from a photograph, AI image or from text — generating the image within Hedra. Then you just click generate video and wait.
There are similarities of functionality to some open source projects, research previews and even lip-synching tools in platforms like Runway and Synclabs — but it is the future promise and expression in the videos that make Hedra stand out for me.
The company said of its future plans: “This is the first step in Hedra’s mission to build a multimodal creation studio accessible for everyone, giving creators complete control over emotional dialogue, movement, and (yes) entire worlds.”
How well does Hedra Character-1 work?
Introducing the research preview of our foundation model, Character-1. Available today at https://t.co/G45zFlUfcN (on desktop and mobile).* Infinite duration (30s for open preview)* 90s generated per 60s (if our H100 supply holds)* Expressive talking, singing, rapping… pic.twitter.com/cYuHpSnqMuJune 18, 2024
This is the first stage of a new model so there were some teething problems, particularly with the overly strict moderation AI but I had no issues with the videos I generated.
It is currently limited to 30 seconds, so if, like me, you have a longer audio clip you’ll need to do it in two sections. It seems to work best with Hedra-generated images but you can upload your own, just make sure it is human-like and forward-facing.
It currently only offers square format videos rather than widescreen or portrait and the resolution is relatively low. But this is a research preview to showcase the capabilities rather than produce production-ready content and it really does show what is to come.
To put it to the test I created a short story of alien invasion. This let me generate four characters — three aliens from the galactic armada and a human general. While compared to human acting it's as wooden as you’d find in a student soap opera — for AI-based lip-synching, it is a massive step up from what I’ve seen before.
More from Tom's Guide
- Apple is bringing iPhone Mirroring to macOS Sequoia — here’s what we know
- iOS 18 supported devices: Here are all the compatible iPhones
- Apple Intelligence unveiled — all the new AI features coming to iOS 18, iPadOS 18 and macOS Sequoia
