Skip to content

Release 0.11.1: benchmark throughput and pipeline improvements - #177

Merged
dylanuys merged 10 commits into
mainfrom
dev
Sep 29, 2026
Merged

dylanuys merged 10 commits into
mainfrom
dev

Conversation

@dylanuys

@dylanuys dylanuys commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Bumps the package to 0.11.1 with faster video and audio preparation and shared benchmark pipelines.

  • Sequential PyAV video decoding: reads the requested frames directly, avoiding whole-file copies and indexing. A matched, warmed 52-input workload improved from 33 → 23 seconds (~1.4×), with identical predictions.
  • Parallel audio preparation: reusable decoder workers reduced a warmed workload from 139 → 93 seconds (~1.5×), with identical predictions.
  • Lower checkpoint overhead: grouped writes improved a matched video workload from 262 → 115 seconds (~2.3×); streamlined persistence and recovery preserve integrity checks and explicit durable saves.
  • Efficient input handling: metadata-based startup validation, augmentation cache reuse, and skipping checkpointed inputs avoid redundant work.
  • Shared pipelines: consolidated runners, prefetching and cache handling; focused transform modules; stage timings for diagnosing remaining bottlenecks.

These are separate, small workloads; gains vary and are not additive. Video decoding accepts readable prefixes without requiring complete frame-count metadata or a readable tail. Existing checkpoints require a matching evaluator to resume.

Validation: automated tests, decoded-frame comparisons across 271 datasets, matched GPU predictions, interruption/recovery checks, and a clean package build.

@dylanuys
dylanuys merged commit 0a5c7ed into main Sep 29, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant