remotion_caption_burn: Remotion path fails 'No frame found at position 5546'; TalkingHead has hardcoded 9000-frame duration

[srikanthapril19-code]

Summary

remotion_caption_burn fails on its Remotion path (the animated per-word caption renderer) with a compositor error, falling back to plain FFmpeg captions only if the caller passes force_ffmpeg: True. On a normal call it errors out:

Compositor error: No frame found at position 5546 for source <...>/clip_with_audio.mp4

The failure is deterministic and input-independent: I hit the identical position 5546 on two different source files — an 8.3 s clip muxed with -c:v copy, and a clean re-encode of the same clip (H.264, keyframe every frame, 205 frames). Because both are ~8.3 s but produce the same errant position, 5546 cannot be derived from the input duration — it points at the composition/tooling, not the video.

Operating system

Windows 11, Python 3.12, Remotion 4.0.484, Node 24

Pipeline

Caption burn-in (remotion_caption_burn), Remotion path (no force_ffmpeg)

Runtime / renderer

remotion

Steps to reproduce

1. Take any short clip (~8 s) with an audio track.

2. Transcribe it with transcriber to get word-level segments.

3. Burn captions without force_ffmpeg:

   registry.get("remotion_caption_burn").execute({
       "input_path": "clip_with_audio.mp4",
       "output_path": "out.mp4",
       "segments": segments,
       "words_per_page": 3,
       "font_size": 64,
   })

Expected behavior

An MP4 with animated per-word captions burned in.

Actual behavior

The npx remotion render TalkingHead subprocess aborts with Compositor error: No frame found at position 5546. No output file is produced. force_ffmpeg: True works but renders static centred captions, losing the animated per-word highlight that is the point of the Remotion path.

Relevant logs or error output

Could not extract frame from compositor Error: Compositor error:
  No frame found at position 5546 for source <tmp>/remotion-assets-dir/<hash>.mp4
  (original source = http://localhost:3000/public/talking-head/clip_with_audio.mp4)
tools.base_tool.ToolCommandError: Command '[... 'remotion','render','TalkingHead',
  '--props=public/demo-props/caption-burn-clip_with_audio.json',
  '--width=1920','--height=1080','--fps=30','--frames=0-249', ...]' returned non-zero exit status 1

Investigation / prime suspect

TalkingHead is registered very differently from its siblings in remotion-composer/src/Root.tsx:

// Root.tsx (~L179)
<Composition
  id="TalkingHead"
  component={TalkingHead}
  durationInFrames={30 * 300}   // hardcoded 9000 frames (5 min), no calculateMetadata
  fps={30}
  width={1080}                  // declared PORTRAIT...
  height={1920}
  ...
/>

Two things stand out versus Explainer / TitledVideo, which both derive duration from content via a calculateMetadata (e.g. durationInFrames: Math.ceil((lastEnd + 1) * 30)):

1. durationInFrames is a fixed 30 * 300 (9000 frames) rather than computed from the clip. The Python side does clamp the render with --frames=0-{ceil(duration*30)-1} (tools/video/remotion_caption_burn.py ~L291/L335), and that math is correct for an 8 s clip (0-249) — but <OffthreadVideo src={videoSrc}> in TalkingHead.tsx (~L320) has no explicit bounds, so the composition's own 9000-frame length may still drive a source seek beyond the clip.

2. Declared dimensions are portrait (1080×1920) but the tool overrides to landscape via --width/--height. Likely unrelated to the seek error, but it means the registered composition never matches the actual render and is worth aligning in the same pass.

Honesty caveat: I could not fully derive why the position is exactly 5546 — I confirmed it is not input-duration-derived (identical across two different-encoding files of the same length) and that the Python frame clamp itself is correct, but I did not trace 5546 to a specific line. The hardcoded 9000-frame duration with no calculateMetadata is the strongest candidate, not a proven root cause.

Suggested fix

Give TalkingHead a calculateMetadata that sets durationInFrames from the actual video (as Explainer/TitledVideo already do), and align its default width/height with how the tool renders it — or have the tool pass dimensions/duration the composition trusts. Happy to test a patch against this exact repro (Remotion 4.0.484).

Reminder for triage, matching a pattern in #431/#432: remotion_caption_burn reports available whenever Remotion is installed, so this only surfaces at call time.

[srikanthapril19-code]

Follow-up — I attempted the suggested fix locally and it refined the diagnosis but did not resolve the crash, so the hardcoded duration is only part of the story.

What I tried (mirroring TitledVideo): added a calculateTalkingHeadMetadata using getVideoMetadata to set durationInFrames = Math.ceil(durationInSeconds * 30) plus the clip's real width/height, wired it into the TalkingHead <Composition>, and switched the registered default from portrait 1080×1920 to 1920×1080.

Result: the composition duration was clearly corrected (the render no longer behaves as a 9000-frame timeline), but the compositor still aborts — and the error position moved rather than disappearing:

before fix:  No frame found at position 5546
after fix:   No frame found at position 13226

Why the position moved, and the likely real cause. The position is a source-media frame index, and it scales with the composition's frame rate relative to the source. My test clip is a Pexels stock video at 25 fps composited into a 30 fps composition. When Remotion's OffthreadVideo maps a 30 fps composition frame onto a 25 fps source, and the composition (or the CLI --frames/--fps overrides the tool passes) is even slightly longer than the source, it requests a source frame beyond the clip's last one. Correcting the duration changed the arithmetic (hence 5546 → 13226) but did not eliminate the over-seek.

So there appear to be two interacting factors:

1. The hardcoded durationInFrames / missing calculateMetadata (addressed by the patch above).
2. A source-fps vs composition-fps mismatch in how the clip is seeked — the tool hardcodes --fps=30 regardless of the source clip's real frame rate, and OffthreadVideo has no explicit end bound.

A robust fix likely needs both: size the composition to the clip and either match the composition fps to the source, clamp the video with an explicit endAt/Sequence bound, or trim the render range to min(compositionFrames, sourceFramesConvertedToCompositionFps).

I reverted my local changes since they didn't fully fix it — didn't want to imply a working patch. Repro clip was 25 fps H.264; happy to share exact ffprobe output if useful. Environment: Remotion 4.0.484, Windows 11, Node 24.