examples/egocentric: fail loudly, not with a raw traceback, when hf is missing - #78
Conversation
…s missing Following the README's documented setup (uv sync --locked --all-extras, then prepare.py) from a clean environment crashes with an uncaught FileNotFoundError: the `hf` CLI that prepare.py shells out to download the pinned corpus is never declared anywhere (pyproject.toml, CONTRIBUTING.md, or either example README) as something you need to separately install. Check for `hf` on PATH before invoking it and raise the existing friendly RuntimeError, and tell both READMEs how to actually get it. Verified against the real pinned corpus (builddotai/Egocentric-10K, factory_051/worker_001): with `hf` installed and authenticated, prepare.py downloads and hash-verifies the archive, generates all 96 episodes (6 with injected faults), and pipeline.py + curate reproduce the README's exact numbers (90 ok / 6 quarantined, black_frame_pct ~15 and freeze_total_s ~3 on the fault episodes, 90-row manifest with 96/96 camera_health coverage).
kstonekuan
left a comment
There was a problem hiding this comment.
Thank you @chintondutta. This is exactly the work I was hoping you would go do: you ran the example end to end against the real pinned corpus, hit a rough edge a new user hits on their very first command, and fixed both halves of it (the crash and the missing instruction that caused it).
What I validated locally:
- Confirmed the root cause:
huggingfaceappears nowhere inpyproject.toml, so a cleanuv sync --locked --all-extrasgenuinely leaves you without thehfCLI thatprepare.pyshells out to. The dependency was real but undeclared and undocumented, which is the actual bug. - Exercised your new guard directly, with
hfforced offPATH, and got the intended message instead of a traceback: "thehfCLI is required to download this dataset but is not on PATH. Install it withuv tool install -U huggingface_hub, thenhf auth login." - Full quality gate is clean (
ruff check,ruff format --check,ty check), 307 tests pass, and lychee reports 230 links with 0 errors. - Checked that
shutilwas already imported inprepare.py, so the guard adds no new import.
Two things I appreciate. First, raising the same friendly RuntimeError shape the file already uses for a failed download, rather than inventing a new error style for this one case. Second, fixing the READMEs in the same pass: "an authenticated hf CLI" reads as a precondition you already satisfy, and uv tool install -U huggingface_hub is the missing sentence that makes it actionable.
Your end-to-end numbers are a useful record too. 96 episodes, 90 ok and 6 quarantined, black_frame_pct around 15 on the three blackout episodes, freeze_total_s around 3 on all six, and a 90-row manifest at 96/96 camera_health coverage. That matches what the README claims, which means the example's documented output is still honest.
Merging now, and thank you for keeping to one pull request at a time. Please keep going in this direction: corpus-level rough edges like this are worth more to the project than any of the starter issues.
The gap
Following the egocentric example's README exactly from a clean environment:
crashes with an uncaught
FileNotFoundError: [Errno 2] No such file or directory: 'hf'.prepare.pyshells out to thehfCLI to download the pinned corpus, but that CLI is never declared as something to install: it's absent frompyproject.toml(main deps, extras, and the dev group), and neitherexamples/README.mdnorexamples/egocentric/README.mdsays how to get it -- both just say "authenticatedhfCLI" as if it already exists on your machine.Fix
prepare.py: checkshutil.which("hf")before shelling out and raise the same style of friendlyRuntimeErrorthe code already uses for a failed download, instead of letting a bareFileNotFoundErrortraceback surface.uv tool install -U huggingface_hub, thenhf auth login).Testing
Reproduced the original crash from a clean environment (no
hfon PATH), confirmed the fix now raises a clear, actionable error instead.Then installed and authenticated
hfand ran the documented workflow against the real pinned corpus end to end:uv run python examples/egocentric/prepare.py uv run python examples/egocentric/pipeline.py data/egocentric/landing/*.mcap uv run hflow curate --catalog data/egocentric/catalog --sql-file examples/egocentric/curate.sql --output data/egocentric/manifest.parquetok, 6quarantined; the three blackout episodes showblack_frame_pct≈ 15, and all six showfreeze_total_s≈ 3 -- exactly what the README describes.camera_healthcoverage 96/96 -- matches the README.All pass.