Repository navigation
ADFA-4894 feat(dashboard): resilient content downloads — books/kiwix hardening + reconnect - #451
Merged
Merged
Conversation
…hardening + reconnect Area 1 of the ADFA-4893 download contract, on the REST job engine. A network drop mid-job now recovers or fails cleanly instead of failing the whole batch at the first blip. - net-retry.ts: a shared withRetry(fn) with exponential backoff + jitter that bails on cancel / AbortSignal; pure and unit-tested (net-retry.test.ts). - jobs.ts: each job carries an AbortController; cancel() aborts it and the runner gets ctx.signal — so cancel now interrupts an in-flight fetch, not just child processes (a fetch has no process to SIGKILL). - books.exec.ts: per-attempt timeout + ctx.signal on both fetches, per-item retry with backoff (retries network / timeout / 5xx, not 4xx), Aborted -> CanceledError so a canceled job lands on 'canceled'. A dropped EPUB no longer fails the whole batch. - kiwix.exec.ts: prune the partial .zim + .aria2 on cancel (clean-on-cancel), and add aria2 reconnect flags (--max-tries=5 / --retry-wait=5 / --timeout=60). The divergence from the app's Aria2Manager (--max-tries=1) is deliberate and documented: the app has an outer retry loop, the REST engine does not, so aria2 owns the reconnect here. maps per-layer checkpoint is deferred — it needs the on-box tile-extract.py output layout, which lives outside this repo. Layer B (the paused / pause / resume / retry control surface) is a separate commit. Version 1.2.3-dev.1 (stays -dev.N through the PR; final 1.2.3 at merge) + CHANGELOG. Verified: tsc --noEmit clean, 79/79 dashboard tests pass (incl. 6 new net-retry).
…books budget, canonical aria2 flags Addresses the two-pass review + the on-device findings on 1.2.3-dev.1. - kiwix: an outer withRetry loop around aria2 recovers a FULL interface loss — aria2 exits 19/DNS and can't retry a name it can't resolve — by re-running (resuming via --continue) on transient exit codes (1/2/6/7/19/29); terminal codes (not found / no space / exists) are not retried. Cancel still cleans the partial; a real error keeps it for a later --continue. - books: retry budget widened (6 tries, 1/2/4/8/15s ≈ 30s) so a real Wi-Fi drop + reassociation survives, not just a sub-4s blip. - aria2 flags: the 4832 sync note is now a canonical checklist (the three mirror sites + the two documented divergences from Aria2Manager), so a silent drift is visible in review. Version 1.2.3-dev.2. tsc --noEmit clean; 79/79 dashboard tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Harden the REST job runners so a network drop mid-download recovers or fails cleanly,
instead of failing the whole batch at the first blip. Area 1 of the ADFA-4893 contract,
on top of the existing durable jobs.ts substrate.
Why
On real, intermittent links a single drop killed a whole content job, cancel could not
interrupt an in-flight fetch, and a canceled kiwix download left its partial behind.
How
that bails on cancel / AbortSignal.
ctx.signal — so cancel interrupts an in-flight fetch, not just child processes.
backoff (retries network / timeout / 5xx, not 4xx); a dropped EPUB no longer fails the
batch; Aborted -> CanceledError so a canceled job lands on 'canceled'.
reconnect flags (--max-tries=5 / --retry-wait=5 / --timeout=60). The divergence from the
app's Aria2Manager (--max-tries=1) is deliberate and documented: the app has an outer
retry loop, the REST engine does not, so aria2 owns the reconnect here.
Out of scope
repo) — deferred.
separate commit/PR, the REST mirror of the APK-side ADFA-5119.
Verification
tsc --noEmit clean; 79/79 dashboard tests pass (incl. 6 new net-retry). Device: a mid-job
network drop recovers (books/kiwix); cancel leaves no kiwix partial.