05 / AI mediaAutomated video production systemDemo

AI Video Factory

An automated AI video production system that turns source material into scripted, narrated, captioned, and composed short-form video.

The pipeline spans trend discovery, transcription, script generation, narration, word-level alignment, captions, avatar generation, visual effects, and final encoding. Below is one build, from brief to finished file.

One build, end to end.

Every stage writes an explicit artifact the next stage reads, so any step can be inspected, swapped, or rerun. This sample runs the narration, timing, and caption layers of the real renderer — the same code, not a mockup.

1 · Brief

A title and format go in: “How the local video pipeline is assembled” — 9 : 16, 1080 × 1920, explainer template, six scenes.

2 · Scene plan

A structured spec (timeline_014.json): per scene, a narration line, a caption headline, a visual intent, and a method.

The scene spec the renderer reads. Planned durations are indicative — the renderer recomputes screen-time from the measured narration length.
#NarrationCaption
01The project is a collection of separate local video stages.Separate stages
02Script generation writes a script and structured beat data.Script and beats
03A voice stage synthesizes narration as its own artifact.Voice artifact
04Alignment data becomes timed karaoke-style captions.Timed captions
05Avatar and visual-effects stages can be added before final composition.Optional visual stages
06The current runner works locally, but still depends on machine-specific paths.Local, not portable

3 · Narration & timing

Each scene's line is synthesized to a standalone audio clip with clause-boundary pauses inserted, then ffprobe measures its real duration. That measured duration — not the planned time in the JSON — becomes the scene's screen-time. Timing is the edit.

4 · Captions & frames

The spoken narration is split into short readable chunks and timed to each chunk's share of the measured voice duration — a dependency-free stand-in for the pipeline's WhisperX forced alignment. One frame is drawn per chunk: scene tint, shot number and timecode, headline, the current caption band, and a baked progress bar.

A rendered vertical frame: ‘SHOT 04, 00:13.3–00:17.4’, a caption band reading ‘alignment data becomes timed karaoke style’, the headline ‘TIMED CAPTIONS’, and a footer showing method ‘GRAPHIC’, 3.6s, voice segment 4.
Scene 4's caption frame, straight out of the renderer.

5 · Assembly

Each frame plus its slice of narration becomes an ffmpeg segment; segments are concatenated and a title card is prepended. Output: a 1080 × 1920, 30 fps MP4.

The finished file for the six-scene spec above — narration, measured timing, and caption cards composited and encoded. The visual-asset stage and the SadTalker avatar branch are part of the wider pipeline but are not run for this public sample; here every scene is its caption card.

System and human.

System handles

The production run

Transcription and word timing (WhisperX), script and beat generation, narration synthesis and pacing, duration measurement, caption chunking, frame drawing, segment assembly, and the final encode.

Human handles

The judgement

Topic and brief, the scene plan, whether a run is good enough to publish, and — for this sample — the call to ship the caption-card render without the avatar.

The stack.

  • Transcription

    WhisperX for transcripts and word-level timestamps.

  • Script

    LLM script and beat generation.

  • Voice

    Neural TTS narration, rendered as a standalone audio artifact.

  • Captions

    Alignment into ASS karaoke captions, burned in at composition.

  • Avatar

    SadTalker talking-head output, composited with effects.

  • Assembly

    FFmpeg for the final composition and encode.