You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8ccb21c
Browse filesBrowse the repository at this point in the historyBrowse files
feat: unify iOS record and stream onto one H.264 capture path
`stream-video` had no native format on iOS: `mjpeg`/`raw`/`ffmpeg` were a
screenshot-per-frame loop and `bgra` was raw pixels, so the only way to
watch a simulator live was ~4 fps at ~1.9 MB/s. Meanwhile `record-video`
had been driving native H.264 since lycorp-jp#52/lycorp-jp#70 — the capability was already
there, just not wired to stdout.
Both verbs now build one `FBVideoStreamConfiguration.h264Capture(...)` and
differ only in their sink:
record-video -> H264MuxingPipeline -> MP4
stream-video -> stdout
That muxer's own docs already described it as "shared by the iOS
(FBSimulatorVideoStream) and Android (adb screenrecord) capture paths";
iOS simply never connected to it, using idb's separate in-process file
writer instead. `AnnexBPipelineConsumer` is the missing adapter from idb's
byte-stream consumer protocol to the pipeline.
Measured on a booted iPhone 17 Pro (iOS 27.0), 6 s captures:
screenshot loop (mjpeg) 24 frames 4.1 fps 11.2 MB
screenshot loop (PNG) 26 frames 4.5 fps 92.8 MB
native h264 Annex B 144 frames 24.0 fps 1.33 MB
The one real trade-off: frame timestamps now come from host arrival time
instead of the encoder's sample clock, since Annex B carries no timing.
Across three runs each under real touch input, inter-frame jitter measured
6.0-6.8 ms on the new path against 2.7-3.1 ms on the old — both far inside
a 33 ms frame at 30 fps, and the `--fps` constant-rate and 100 ms-grace
SIGTERM E2E tests both still pass.
fmp4 transport was evaluated first and rejected: idb's FBFMP4FrameWriter
emits live fragments with no `sidx`/`mfra`, which AVFoundation reads as
zero frames, so a recording would have been unplayable in QuickTime and
untranscodable to GIF. Annex B keeps the MP4 a regular non-fragmented file.
Verified live: both `stream-video --format h264` surfaces plus the
top-level route on an iOS UDID, `ffmpeg -f h264` remux, `bgra`
non-regression, the GIF path, and the full iOS stream-video and
record-video E2E suites.
Refs lycorp-jp#132
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: onevcat <onevcat@gmail.com>
Copy file name to clipboardExpand all lines: CHANGELOG.md
+9Lines changed: 9 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
8
8
## [Unreleased]
9
9
10
+
### Added
11
+
12
+
-`stream-video --format h264` now works on iOS, not just Android: a native `FBVideoStream` H.264 Annex B passthrough copied straight to stdout with no host-side codec pass. On a booted iPhone 17 Pro this delivers ~24 fps at ~1.3 MB per 6 s, against the screenshot loop's ~4.1 fps at ~11 MB — roughly 5.9x the frame rate for an eighth of the bytes. It consumes exactly like the Android leg: `sim-use stream-video --format h264 --device $UDID | ffplay -f h264 -probesize 32 -fflags nobuffer -`. The top-level `stream-video` no longer rejects `h264` for iOS targets, so the only platform-exclusive format left is the experimental iOS-only `bgra`. (#132)
13
+
14
+
### Changed
15
+
16
+
- iOS `record-video` and `stream-video` now drive the same capture. Both build one `FBVideoStreamConfiguration.h264Capture(fps:quality:scale:)` and differ only in where the encoded bytes land — `record-video` into the shared `H264MuxingPipeline` → MP4 (the muxer Android has always used, and which was written for both), `stream-video` into stdout. Recording no longer goes through idb's separate in-process file writer. Output is equivalent: the MP4 stays a regular non-fragmented file that QuickTime, Finder and `GIFTranscoder` all read, `--fps` is still honoured as a constant rate, and the finalize path still survives a 100 ms-grace SIGTERM across repeated runs. Frame timestamps now come from host arrival time rather than the encoder's sample clock, which measures as ~6 ms of inter-frame jitter against the previous ~3 ms — both far inside a 33 ms frame at 30 fps. (#132)
17
+
18
+
10
19
### Fixed
11
20
12
21
- The bundled skill preflight now rejects an older `sim-use` CLI before device discovery and prints the Homebrew upgrade command, instead of misdiagnosing newly documented device types as disconnected.
@Option(help:"Output format: h264 (native passthrough on both platforms — fastest, recommended); mjpeg, raw, ffmpeg (screenshot-backed, deprecated); bgra (iOS-only raw pixels). Default: mjpeg")
45
45
varformat:OutputFormat=.mjpeg
46
46
47
-
@Option(help:"Frames per second (1-30, default: 10). Ignored by --format h264 (native variable frame rate).")
47
+
@Option(help:"Frames per second (1-30, default: 10). On iOS --format h264 records at this constant rate; Android's h264 ignores it (native variable frame rate).")
throwCLIError(errorDescription:"--format h264 is Android-only for now (iOS H.264 passthrough is a separate follow-up). Use record-video for an H.264 file, or mjpeg/raw/ffmpeg to stream.")
0 commit comments