Media & entertainment · Generative video production
From a script to on-model animated scenes, with every shot versioned, checked and costed
A media publisher had an original kids’ series and no animation studio. We treated generative video as an engineering problem: a show bible compiled into every prompt, an AI reviewer calibrated against a human, and a cost log behind every frame.
1/8 → 8/8
Keyframes on-model before and after the first rule revision
2 tiers
Standard and Premium cuts of the same scene, each with its measured cost per finished minute
½ ¢
Cost of each automated quality check by a vision model
< 2 weeks
From a word-processor script to a finished pilot scene and a working production system
01
The problem
The client owned an original preschool series: a cast of animal characters, renders made in a consumer AI tool, a few seconds of recorded voice per character and a first script in a word-processor file. They wanted a weekly animated episode at a predictable price. Traditional animation was out of budget.
Their own experiments with consumer AI video showed the real problem. Characters drifted from shot to shot, two characters ended up sharing one voice, and the set changed every time the camera moved. Nobody could say what an AI-produced minute actually costs, how many attempts a shot needs or how much human review it takes.
02
Why it was hard
- Character consistency across shots and engines. Early keyframes invented extra animals, and accessories changed mid-clip. Every video engine also has its own look.
- Talking animals with the client’s voices. Most lip-sync models exclude non-human faces, and the engine with the best picture invented its own voice and mispronounced a lead character’s name.
- Spatial continuity. Backgrounds contradicted each other between camera angles: furniture moved, props multiplied, the wrong wall appeared behind a character.
- The AI reviewer itself was unreliable. Vision checks were noisy between runs and missed small background details until they were calibrated against human judgement.
- Cost and vendor churn: credit-based pricing across vendors, features locked behind enterprise plans and new model versions released mid-project.
03
What we built
A file-based production pipeline directed by an AI orchestrator, where every stage runs as code and a human approves at three gates. Nothing reaches the producer without going through the rules, the automated review and the edit.
04
Where the innovation is
01
Prompts are compiled, never written by hand
Each character has a spec: details that must never change, how it moves, its real-world scale, how its name is pronounced. Together with the show’s rules for camera, casting, physics and set design, it is compiled into every prompt. Each rule change is a new version with a log of what changed, why, the result and the cost. The knowledge lives in the project, not in a model, so it survives any vendor change.
02
Every rejection becomes an executable rule
When the producer rejects a take, the reason becomes a rule, an automated check and a regression case. The build fails if any rule has no check covering it. The pipeline learns from every rejected frame instead of repeating the mistake on the next shot.
03
An AI reviewer, calibrated against a human
Quality review is split into narrow questions, because a narrow question gets a better answer: who is in the shot, is each character on-model, does the mouth match the line, are the spoken words right, does the set match the previous angle. The reviewer model was chosen by comparing its verdicts with a human’s, and where vision models judge poorly, classic computer vision measures instead: camera motion is tracked from background features and color grade is checked against neighboring shots.
04
The client’s voice first, the picture fits around it
The cloned voice is locked before any video exists, within about 1% of the original speaker’s pitch. When the engine with the best picture invented its own timing, each client line was aligned and stretched by a few percent onto that engine’s mouth movement. The picture stays untouched, and the client’s voice and words stay exact.
05
The same room, by construction
Image models redraw a set slightly differently from every angle. After testing alternatives, including a 3D blockout, the approach that held was one continuous camera move generated from the approved set, with every other angle taken from that single shot. Consistency comes from how the set is made, not from hoping each prompt agrees.
06
The cheapest fix that can work
A repair router sends each defect to the least expensive fix that can converge: a local color correction for look issues, a targeted edit for small defects, a full regeneration for geometry. Paid video is blocked until every still is locked. That rule exists because a batch of video was once generated too early.
05
What the measurements showed
Every take, check and paid call was logged with its full recipe. That turned “AI animation” from a demo into something with unit economics, and made the reviewer’s weak spots visible before they reached the client.
Attempts per approved shot
Measured during the pilot. Premium uses a more cinematic engine with the client’s voice fitted afterwards.
Agreement between the AI reviewer and the human producer
Calibration rounds during the build. Lip sync was the weak spot, so it was recalibrated instead of trusted.
06
Results
- A complete pilot scene in two quality tiers, Standard and Premium 1080p, both produced end to end by the pipeline.
- A show catalog ready for production: character specs, model sheets, and the client’s sets rebuilt as consistent multi-angle sets.
- Unit economics measured instead of guessed: about one retry every three shots at Standard and about two attempts per shot at Premium, with every paid call logged.
- AI spend to reach production quality in the hundreds of dollars, not thousands, and about half a cent per automated quality check.
- A script that ran almost three times its target length was caught on day one, before a single frame was rendered.
- The data became the basis of a two-tier, per-minute pricing model and a fixed-price pilot proposal.
Camera judder, before and after measured correction
Share of frames with judder across the tested shots, measured by tracking background features, not by asking a model.
07
What we learned along the way
One word in the style line
The lead character kept looking like a plush toy. The cause was a single word in the global style description, compiled into every prompt since the first version. Because prompts were compiled and versioned, it took one change to fix every future shot.
No shortcuts reach the producer
A rushed test in a vendor’s own app skipped the rules, the review and the edit, and it showed: two characters with the same voice and rough cuts. Since then, nothing reaches a human reviewer without the full pipeline, whatever the engine.
Knowledge in the project, not in the weights
Fine-tuning local models to “remember” what works was proposed. The answer was that the versioned rules already are the memory, and unlike model weights, they move with the project to any vendor.
Stack
- Node.js
- Python
- ffmpeg
- Computer vision (feature tracking, RANSAC)
- Blender
- SQLite + vector search
- Gemini image and vision models
- Video models (Veo, Kling, Seedance and others)
- Voice cloning
- Speech recognition
- Claude Code (orchestrator + subagents)
- Vercel
Is AI almost good enough for your creative process?
We turn it into a governed pipeline: rules you can version, checks you can trust and a cost you can predict before you scale.
Book a discovery sessionClient identity, characters, region and commercial terms are withheld. No frames of the client’s characters or sets are shown, and the internal workings of the pipeline are described at the level of method only. Delivered as the specialist technology partner of a production and development agency, which held the direct client relationship.