Summary
XetFileDownloadGroup.abort() returns immediately and makes wait_to_finish() raise, but the group's in-flight transfer keeps downloading at full bandwidth in the background until the entire file would have completed. The bytes are then discarded (no chunk cache growth, no output file).
Repro
Via the Python bindings (hf_xet 1.6.0, Linux x86_64): start a download of a 1.07 GB file (unsloth/Qwen3-1.7B-GGUF, Qwen3-1.7B-Q4_K_M.gguf) through XetSession.new_file_download_group(), then call group.abort() from another thread 5 seconds in, while the main thread blocks in wait_to_finish().
Observed (network RX sampled from /proc/net/dev, ~6.7 MB/s pipe):
t=5s group.abort() called; wait_to_finish raises ~0.1s later
t+10s..t+180s sustained ~67 MB per 10s received (full pipe)
t+190s traffic drops to idle
Total transferred after abort: roughly 1.2 GB, matching the full file plus overhead. The xet chunk cache stayed at 2 MB the whole time, so the data was fetched and thrown away.
For comparison, XetSession.sigint_abort() does stop traffic promptly, but it tears down the whole session and every other concurrent download with it.
Expected
abort() cancels the group's outstanding chunk transfers, or at least stops scheduling new ranges, so the bandwidth is actually released to other downloads in the session.
Suspected area
FileDownloadGroup::abort() in xet_pkg/src/xet_session/file_download_group.rs does task_runtime.cancel_subtree() and aborts the download join handles, which unblocks the Python caller. But the chunk range fetches appear to be owned by the session-wide transfer scheduler rather than the group's task subtree, so cancellation never reaches them and they run to completion.
Context
Hit while wiring per-download cancellation into huggingface_hub.hf_hub_download (huggingface/huggingface_hub#4632). The group-level isolation works as documented (the session stays usable and other downloads are unaffected); it is only the background bandwidth of the aborted group that leaks.
Summary
XetFileDownloadGroup.abort()returns immediately and makeswait_to_finish()raise, but the group's in-flight transfer keeps downloading at full bandwidth in the background until the entire file would have completed. The bytes are then discarded (no chunk cache growth, no output file).Repro
Via the Python bindings (
hf_xet1.6.0, Linux x86_64): start a download of a 1.07 GB file (unsloth/Qwen3-1.7B-GGUF,Qwen3-1.7B-Q4_K_M.gguf) throughXetSession.new_file_download_group(), then callgroup.abort()from another thread 5 seconds in, while the main thread blocks inwait_to_finish().Observed (network RX sampled from
/proc/net/dev, ~6.7 MB/s pipe):Total transferred after abort: roughly 1.2 GB, matching the full file plus overhead. The xet chunk cache stayed at 2 MB the whole time, so the data was fetched and thrown away.
For comparison,
XetSession.sigint_abort()does stop traffic promptly, but it tears down the whole session and every other concurrent download with it.Expected
abort()cancels the group's outstanding chunk transfers, or at least stops scheduling new ranges, so the bandwidth is actually released to other downloads in the session.Suspected area
FileDownloadGroup::abort()inxet_pkg/src/xet_session/file_download_group.rsdoestask_runtime.cancel_subtree()and aborts the download join handles, which unblocks the Python caller. But the chunk range fetches appear to be owned by the session-wide transfer scheduler rather than the group's task subtree, so cancellation never reaches them and they run to completion.Context
Hit while wiring per-download cancellation into
huggingface_hub.hf_hub_download(huggingface/huggingface_hub#4632). The group-level isolation works as documented (the session stays usable and other downloads are unaffected); it is only the background bandwidth of the aborted group that leaks.