OmniHuman by ByteDance

OmniHuman starts from one image, not a rig, a motion-capture suit or a video of the actor. It accepts any aspect ratio and any body proportion, so a face close-up, a waist-up portrait and a head-to-toe shot all work without cropping. The body moves as one piece rather than splitting into a floating head over a still torso. For creators, that means the shot you already have is often enough to begin.

The model reads an audio track and produces more than moving lips. This audio-driven talking head moves the whole body: speech drives gestures, posture and small idle movements that fit the rhythm of the voice. Singing works too, including high notes and the pauses between phrases, and the motion shifts with the music style. This is the part that separates OmniHuman from a plain lip-sync filter, which only ever moves the mouth.

Because the model trains on mixed motion signals, it does not stop at audio. It can follow a reference video to copy specific body actions, and it can blend audio with video to control details like a particular gesture while the voice drives the delivery. That flexibility comes from the training approach, where motion conditions are mixed rather than split into separate specialized models.

The 1.5 version adds a reasoning step that plans motion before it renders. A Multimodal Large Language Model reads the image, the audio and any text prompt, then writes a structured plan of what the character should do and when. The renderer follows that plan, so a line about reading a letter becomes a sequence of reach, open and look, not a random gesture. The team calls this pairing System 1 and System 2, and borrows the terms from the psychology of fast and slow thinking.

You can write a prompt to steer the scene. Prompts in the demos cover camera moves like a slow pan or a handheld shot, plus actions such as turning to face the lens or reaching toward it. The model keeps audio sync while it follows the instruction, so the character still mouths the right words as the camera moves. For anyone planning a shot, that turns a text line into a first draft of the frame.

OmniHuman-1.5 handles scenes with more than one person by sending each audio track to the right character in the same frame. Group dialogue and ensemble shots work without cutting between separate clips. The model also covers stylized inputs, from cartoons and 3D characters to real animals, and it matches the motion to each subject's own style rather than forcing a human template onto it.

The 1.5 model generates clips over a minute long with continuous camera movement and complex interaction, which is well past the few-second loops that many avatar tools stop at. The team's own demonstrations show dynamic motion across that length rather than a static pose that slowly drifts. Longer clips matter for storytelling, where a scene needs to breathe instead of resetting every few seconds.