TWO RIGS, FROM SCENARIO MODELS · A FILM THEY MADE POSSIBLE
A man was drawn, built and rigged from Scenario models, with a face rig and a body rig. Then he acted: every move, glance and word was staged in a game engine, and a video model only finished the picture. The film is what the rigs made possible.
01 / SAME TAKE, TWO WORLDS
On the left, the blocking: gray corridors in Unreal Engine, a rigged actor, camera moves and lamp states chosen by hand. On the right, the picture a video model finished on top of it. The cuts, the timing and the movement stay put while the world changes around them. Where the model was asked to edit, the geometry holds exactly. Where it was asked to recreate the shot, it sometimes re-framed: the opening close-ups and the fear shot came out tighter than the blocking. The shots from 6 to 13.5 seconds were finished a second time with MiniMax H3, from finished keyframes that share one corridor, after the first pass left them with blotchy walls and pale skin.
Sound on for the voice and the room tone. The mouths in 10 to 12 seconds and 16 to 19 seconds carry a lip-sync pass; the rest are the model's own. Download the film ↓
02 / THE RIGS
This exploration is about the rigs. A character sheet became a body rig and a face rig through a chain of Scenario models, each adjusted by hand where it fell short. The film that follows is the result of that work.
An image model drew four character sheets of a man in his fifties. One was chosen: burgundy shirt, rolled sleeves, a long face and a crooked nose. The body was then redrawn with a blank sphere for a head, and the head on its own, so each part could be turned into 3D at full detail.
An image-to-3D model built the body and the head separately. An automatic decimation pass smeared the head texture, so a second 3D service rebuilt both meshes while keeping the original UVs and colors. The parts were then fitted together in Blender and loaded into Unreal Engine.
An auto-rig produced 92 bones with Unreal Mannequin names, fingers, twist bones and IK, plus exactly 52 ARKit face shapes. The jaw skinning was repaired by smoothing weights in space instead of across the mesh, so the beard and hair shells move together with the lip seam kept sharp.
Before one frame of the film existed, the face shapes were driven one by one inside Unreal Engine: jaw, funnel, pucker, smile, frown, looks and blinks. Fear, tension and tiredness are combinations of these shapes, not painted on later.
Every step of the actor went through a Scenario model. This is what each one did, and what we had to adjust by hand.
The first skin let the beard and hair shells lag behind the chin. Weights were smoothed by distance in space instead of along mesh edges, so every shell moves together, and the lip seam was kept sharp. The jaw opens up to 62 millimeters, and speech runs it at a fraction of that, with a gain between 0.36 and 0.62 depending on the take.
Body: 147,778 triangles, no unweighted vertex, at most four bones per vertex, weights normalized. Head: 114,887 triangles with exactly 52 face shape keys. Eyeballs, upper and lower teeth and tongue are separate meshes.
A pose was rejected because the arm's X axis is the bone's twist. In this rig the upper arm lifts forward on Y and the elbow folds on Z, mirrored on the other side. The neck turns on local Y and nods on X. The spine leans forward on negative X.
The face tore when two keys landed on the same frame and the curve overshot. The face is now sampled every frame, summed and clamped, and a blink cancels squint and wide eyes.
The FBX import brought no textures, so the PBR materials were rebuilt in the engine. 4K textures filled the video memory and made patchy renders, so everything went to 2K except head skin and eyes.
A script reads the voice track: loudness opens the jaw, low-energy sound rounds the lips, brighter sound spreads them. It runs per line, with a jaw gain tuned per take.
The film uses library walks and a hand-built pose, not motion capture. Afterwards the same rig was used to test Viggle's text to motion and video to motion models, which export a Mixamo skeleton.
A Mixamo profile was added to the retarget script and a test clip landed on all 52 mapped bones with none missing. The bone directions matched the source exactly.
Directions matched but the hands were rolled a quarter turn, because the rig's rest pose differs from the T-pose by about 84 degrees. Adding that twist to the whole arm chain brought the palm error from 83 to 1.4 degrees.
With the arm raised, the skin between neck and shoulder dents because the collarbone carries almost no weight on the trapezius. Moving weight, smoothing wider and damping the collarbone each made it worse. It is not repaired.
03 / THE FACE RIG, FROM AUDIO
The star of this exploration is a facial rig that speaks from audio alone. Below, the real rig, with the same 52 ARKit shapes used in the film, is driven by the 30-second voice track and nothing else. Every number on the right is read from the audio and sent to a shape on the left.
The voice track is read 30 times a second in short windows. Three measurements matter in each window: how loud it is against the track's own loud peaks, how much of the sound sits below 700 hertz, and where the center of the spectrum is.
The jaw follows the loudness, raised to a gentle power and smoothed over three frames, then scaled by a per-take gain. The mouth is placed one frame ahead of the sound, because a mouth that opens with the sound looks late.
Low, warm sound rounds the lips: funnel and pucker. Bright sound spreads them: stretch. Silence closes them. The lower lip drops and the upper lip lifts with the jaw, and the cheeks squint a little.
The brows rise on whole phrases, delayed and small, and are pulled down while the jaw is open. Gaze drifts slowly down and sideways. Blinks are the only thing placed by hand, in time.
Twenty-seven of the head's 52 ARKit shape keys are driven here at 30 frames per second. In the clip above the jaw averages about a tenth of its range and peaks near 0.43, with a gain of 0.5. The film used gains between 0.36 and 0.62 depending on the take.
There are no phonemes: closed sounds such as p, b and m are not closed on purpose, and the mouth never reads the words. Reviewers judged the result by eye. It is a rule set on top of a good rig, not speech recognition.
The finished film's lips came from the video model, and they drifted: the generated voice trailed the reference track by about 0.2 seconds when measured on the audio, and reviewers saw mouths still moving after a line had ended. A dedicated lip-sync pass fixed that where the mouth is clearly visible. It was applied only to the shots that were approved, because on a profile or a head from behind it looked wrong.
04 / THE DIRECTOR'S CONTRACT
Most AI video starts with a sentence and hopes. This one started with a performance. The model got a hard split of jobs, and every time it crossed the line the shot was repaired instead of accepted.
05 / BLOCK THE PERFORMANCE
Unreal Engine 5.8 was the stage. The result is deliberately rough, but it makes every decision a director makes: where the camera stands, how he moves, when the light fails and what the room looks like as it gets worse.
Never the whole corridor and the face at once. The camera sits inside the hallway, drops to about 25 centimeters above his forehead for the thinking shots, and backs away in front of him for the walk.
The walk is a retargeted clip, sampled slow with head glances added on top. The last pose, crouched with both hands open, was built by hand after the arm axes were corrected. Three text-to-motion services were compared and none replaced it.
Each lamp has a state, and a procedural dirt material rises across the walls, floor and ceiling as the fear grows. The corridor decays while he walks through it.
The face was torn at first. The cause was duplicate keys on the same frame in the face curves. Sampling, summing and clamping every curve per frame fixed it.
06 / FINISH, DON'T INVENT
Pick a moment. The left frame is the blocking, the middle is the finished still it was paired with, the right is the same instant in the film. The model had both as references and the animatic as a video.
The 30-second blocking went in as the reference video, so shot order, cuts and timing were never up for discussion.
Treat @video1 as a rough director's blocking previs: it fixes the shot order, the cuts and their timing, the shot sizes and general composition of each shot.
Frames of the blocking were finished as stills with an image model, keeping each camera and pose, and passed as references. Hair color, skin tone and the corridor's look came from these frames. For the shots from 6 to 13.5 seconds, every frame came from one picture of the corridor, with only the man changed from frame to frame.
Hair went brown, silver, blue and white; skin turned pale when he looked up. The cause was the animatic's own lighting, which the model copied faithfully. Feeding the blocking as grayscale removed the color it kept copying, and the finished keyframes supplied the right one.
Continuity broke when each shot reinvented the walls. Every shot then received the identical paragraph about the corridor and the same two environment references, and the edit mode locked the geometry.
EDIT @video1. It is a grayscale 3D render... Keep the camera movement, framing, perspective, the geometry and the man's body and head motion EXACTLY identical in every frame; never change, widen, morph, add or remove architecture; only re-finish it.
When one shot failed (a corridor too wide, a mouth that did not speak, a background that morphed) only that shot was generated again from its own animatic frames and spliced in at the same cut points. The last round did this for the walk from behind and the top-down shot, with MiniMax H3. Audio and timing never moved.
The lip-sync model ran on whole shots, and a director chose where to keep it: close and three-quarter faces yes, a profile no. The film was upscaled to 1080p, and the two reworked shots were spliced into that 1080p cut.
07 / WHAT BROKE
It took many passes, most directed by hand with technical terms. These are the real failures, in the order they showed up, and what each one taught.
An edit-style prompt on the animatic returned an almost identical video. Asking for the final film, with the animatic as the director, produced a real finish.
A reference video of exactly 30 seconds was rejected as too long; 29.7 worked. Reference images must sit between 0.4 and 2.5 in aspect ratio, so the 3:1 turnaround sheet had to be split in two.
Not the model: the animatic's lighting. Grayscale input and finished keyframes fixed it, after three prompt rewrites that did not.
Generated mouths kept moving after a line had ended, and the generated voice trailed the reference by about 0.2 seconds. Face-landmark measurements on a stylized face were too noisy to settle the exact lag, so reviewers judged by eye. A lip-sync pass fixed the frontal shots.
In edit mode, smooth animatic walls were painted with large cloud-like blotches. Smoothing the input did not remove them. What finally worked was not editing at all: finished keyframes of one dressed corridor as references, and the animatic only for motion.
On the over-the-shoulder shot the sync looked wrong and was removed.
The first MiniMax and Seedance tests used a first and a last frame whose corridors did not match. The models had to turn one into the other, and the walls faded. The fix was ours: make every frame of a shot from the same corridor picture and change only the man.
Asked to recreate the film, the model re-framed: it pushed in on the opening close-ups and the fear shot and shifted the man in the dolly. Edit mode held the framing exactly but painted blotchy walls. That trade-off ran through the project until the last round, when reference images of one corridor plus the animatic as a motion reference gave both.
In a test clip with a raised arm, the skin between neck and shoulder dents: the collarbone carries almost no weight on the trapezius. Three attempted repairs each made it worse, so it is not fixed. The film's shots keep the arms low.
The hair in the over-the-shoulder shot around 14 seconds still reads like modeling clay. In the reworked shots the corridor still shifts a little from frame to frame, the hair in the walk from behind is lighter than elsewhere, and the pace follows the animatic only roughly.
KEEP EXPLORING
Made with GPT Image 2, Meshy 7.1, Astra 3D, Gemini 3.8 Flash text-to-speech, Seedance 2.5, MiniMax H3, Sync Lipsync 2 Pro and Topaz Video Upscale through Scenario, with the stage built in Unreal Engine 5.8 and Blender.
