fix: bundle CA certificates in the container image - #754
Closed
brawer wants to merge 1 commit into
Closed
Conversation
librqbit's internal HTTP(S) client (used to fetch the planet .torrent file, and its https:// webseed mirrors) builds its own, unconfigured reqwest::Client -- confirmed in its source, no hook to inject a preconfigured one. That falls through to reqwest's actual default in this version: rustls-platform-verifier, which on generic Linux calls rustls_native_certs::load_native_certs() to read the OS's system trust store. There is none in a FROM scratch image -- confirmed live during the first real full-planet smoke test, crashing import_osm outright with "No CA certificates were loaded from the system", seconds in. Our own main.rs::build_client() (used for the ATP fetch, which is why that step worked fine) sidesteps this by explicitly bundling webpki_roots and calling .use_preconfigured_tls() -- but that client isn't, and can't be, passed into librqbit's Session::new_with_opts(). Fix: apk add ca-certificates in the Alpine builder stage, then COPY the resulting bundle (confirmed empirically to land at /etc/ssl/certs/ca-certificates.crt, a real 3071-line Mozilla-style bundle) into the final scratch image at that same path -- rustls_native_certs checks that path among Linux's standard locations. Verified: rebuilt the container locally and ran it against a real workdir. It got well past the exact point that failed before -- import_atp completed real HTTPS transfers, hundreds of MB, no CA error at all. Never reached import_osm in the test window, but only because this sandboxed dev machine's own network path was too slow for the full AllThePlaces dataset (real Hetzner hardware fetches it in ~3.5 minutes; this took over an hour and was still going) -- an environment limitation, not a signal about this fix. Full confirmation deferred to the next real Hetzner full-planet run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 23, 2026
Contributor
Author
|
Closing this in favor of a different fix: #755 removed the entire code path (librqbit's internal, unconfigured HTTP client) that this PR's CA-certificate bundling was working around -- once BitTorrent is gone, there's no more code in Documented properly instead in #757, so this deliberate choice (Mozilla's trust store, not the OS's -- same approach Firefox takes) doesn't get rediscovered the hard way again. |
Contributor
Author
brawer
added a commit
that referenced
this pull request
Aug 24, 2026
#755 removed the last code path (librqbit's internal, unconfigured HTTP client) that ever depended on an OS-provided certificate store -- the only reqwest client left in the codebase already bundles Mozilla's own CA root bundle at compile time via webpki-roots (main.rs's build_client()), same as it always has. That was never written down anywhere, though, and #754 (bundling ca-certificates.crt into the scratch image instead) showed it's easy to reach for "give the container an OS trust store" without realizing there's a deliberate, existing alternative already in place, closer to what browsers like Firefox do. Adds a paragraph to docs/SUPPLY_CHAIN_SECURITY.md's "Minimal containers" section explaining this, right next to where it already covers what else the scratch image deliberately lacks (shell, libc, package manager). Also declares it as an explicit, queryable fact in the CBOM (scripts/sbom/pipeline.jq), extending the existing crypto:tls:* custom-property convention with crypto:catrust:source and crypto:catrust:osTrustStoreUsed on the osm-diffs component -- the webpki-roots crate itself already shows up as an ordinary SBOM library component (cargo-cyclonedx walks Cargo.lock), but that's just one more dependency among ~390 others, easy to miss; this makes "what CA trust store does this software use" answerable directly instead of inferred from a dependency's free-text description. Not modeled as a CycloneDX "certificate" cryptographic-asset: that type's cryptoProperties (subjectName, issuerName, notValidBefore/After, ...) describe a single certificate, not a curated bundle of ~140 unrelated root CAs. Also drops the pre-existing cdx:cbom:version property while in here: checked CycloneDX's actual property taxonomy (github.com/CycloneDX/cyclonedx-property-taxonomy) -- the cdx: namespace is reserved for officially registered CycloneDX properties only ("cbom" isn't among them: registered cdx sub-namespaces are ai-ml, composer, device, esbuild, gomod, lifecycle, maven, npm, pipenv, poetry, python, rustc, plus the standalone cdx:reproducible). Worse, there's nothing for a "CBOM version" to mean in the first place: grepped the actual CycloneDX 1.6 JSON schema (github.com/CycloneDX/specification) and "cbom" appears zero times in it. CBOM isn't a distinct document type or section with its own version -- it's just a CycloneDX document containing one or more cryptographic-asset components (which this file already has, e.g. crypto/protocol/tls-1.3). That property was a fossil from before IBM Research's original, standalone CBOM spec (github.com/IBM/CBOM, which did have its own version number) got absorbed into CycloneDX 1.6 as native cryptographic-asset support. Also drops a stale comment in build_client() referencing librqbit, which #755 already removed. Verified: cargo test/clippy/fmt clean. Regenerated a dev SBOM locally and validated it with cyclonedx-cli validate --fail-on-errors (same check test-container.yml runs in CI) -- passes, with the new properties landing on the osm-diffs component as expected and cdx:cbom:version gone. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
brawer
added a commit
that referenced
this pull request
Aug 24, 2026
#755 removed the last code path (librqbit's internal, unconfigured HTTP client) that ever depended on an OS-provided certificate store -- the only reqwest client left in the codebase already bundles Mozilla's own CA root bundle at compile time via webpki-roots (main.rs's build_client()), same as it always has. That was never written down anywhere, though, and #754 (bundling ca-certificates.crt into the scratch image instead) showed it's easy to reach for "give the container an OS trust store" without realizing there's a deliberate, existing alternative already in place, closer to what browsers like Firefox do. Adds a paragraph to docs/SUPPLY_CHAIN_SECURITY.md's "Minimal containers" section explaining this, right next to where it already covers what else the scratch image deliberately lacks (shell, libc, package manager). Also declares it as an explicit, queryable fact in the CBOM (scripts/sbom/pipeline.jq), extending the existing crypto:tls:* custom-property convention with crypto:catrust:source and crypto:catrust:osTrustStoreUsed on the osm-diffs component -- the webpki-roots crate itself already shows up as an ordinary SBOM library component (cargo-cyclonedx walks Cargo.lock), but that's just one more dependency among ~390 others, easy to miss; this makes "what CA trust store does this software use" answerable directly instead of inferred from a dependency's free-text description. Not modeled as a CycloneDX "certificate" cryptographic-asset: that type's cryptoProperties (subjectName, issuerName, notValidBefore/After, ...) describe a single certificate, not a curated bundle of ~140 unrelated root CAs. Also drops the pre-existing cdx:cbom:version property while in here: checked CycloneDX's actual property taxonomy (github.com/CycloneDX/cyclonedx-property-taxonomy) -- the cdx: namespace is reserved for officially registered CycloneDX properties only ("cbom" isn't among them: registered cdx sub-namespaces are ai-ml, composer, device, esbuild, gomod, lifecycle, maven, npm, pipenv, poetry, python, rustc, plus the standalone cdx:reproducible). Worse, there's nothing for a "CBOM version" to mean in the first place: grepped the actual CycloneDX 1.6 JSON schema (github.com/CycloneDX/specification) and "cbom" appears zero times in it. CBOM isn't a distinct document type or section with its own version -- it's just a CycloneDX document containing one or more cryptographic-asset components (which this file already has, e.g. crypto/protocol/tls-1.3). That property was a fossil from before IBM Research's original, standalone CBOM spec (github.com/IBM/CBOM, which did have its own version number) got absorbed into CycloneDX 1.6 as native cryptographic-asset support. Also drops a stale comment in build_client() referencing librqbit, which #755 already removed. Verified: cargo test/clippy/fmt clean. Regenerated a dev SBOM locally and validated it with cyclonedx-cli validate --fail-on-errors (same check test-container.yml runs in CI) -- passes, with the new properties landing on the osm-diffs component as expected and cdx:cbom:version gone. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
brawer
added a commit
that referenced
this pull request
Aug 24, 2026
#755 removed the last code path (librqbit's internal, unconfigured HTTP client) that ever depended on an OS-provided certificate store -- the only reqwest client left in the codebase already bundles Mozilla's own CA root bundle at compile time via webpki-roots (main.rs's build_client()), same as it always has. That was never written down anywhere, though, and #754 (bundling ca-certificates.crt into the scratch image instead) showed it's easy to reach for "give the container an OS trust store" without realizing there's a deliberate, existing alternative already in place, closer to what browsers like Firefox do. Adds a paragraph to docs/SUPPLY_CHAIN_SECURITY.md's "Minimal containers" section explaining this, right next to where it already covers what else the scratch image deliberately lacks (shell, libc, package manager) -- and a short pointer line from docs/SECURITY.md, matching that file's existing pattern of one-line routes to the more detailed docs (it already does this for SUPPLY_CHAIN_SECURITY.md and TESTING.md) rather than duplicating the explanation. Also declares it as an explicit, queryable fact in the CBOM (scripts/sbom/pipeline.jq), extending the existing crypto:tls:* custom-property convention with crypto:catrust:source and crypto:catrust:osTrustStoreUsed on the osm-diffs component -- the webpki-roots crate itself already shows up as an ordinary SBOM library component (cargo-cyclonedx walks Cargo.lock), but that's just one more dependency among ~390 others, easy to miss; this makes "what CA trust store does this software use" answerable directly instead of inferred from a dependency's free-text description. Not modeled as a CycloneDX "certificate" cryptographic-asset: that type's cryptoProperties (subjectName, issuerName, notValidBefore/After, ...) describe a single certificate, not a curated bundle of ~140 unrelated root CAs. Also drops the pre-existing cdx:cbom:version property while in here: checked CycloneDX's actual property taxonomy (github.com/CycloneDX/cyclonedx-property-taxonomy) -- the cdx: namespace is reserved for officially registered CycloneDX properties only ("cbom" isn't among them: registered cdx sub-namespaces are ai-ml, composer, device, esbuild, gomod, lifecycle, maven, npm, pipenv, poetry, python, rustc, plus the standalone cdx:reproducible). Worse, there's nothing for a "CBOM version" to mean in the first place: grepped the actual CycloneDX 1.6 JSON schema (github.com/CycloneDX/specification) and "cbom" appears zero times in it. CBOM isn't a distinct document type or section with its own version -- it's just a CycloneDX document containing one or more cryptographic-asset components (which this file already has, e.g. crypto/protocol/tls-1.3). That property was a fossil from before IBM Research's original, standalone CBOM spec (github.com/IBM/CBOM, which did have its own version number) got absorbed into CycloneDX 1.6 as native cryptographic-asset support. Also drops a stale comment in build_client() referencing librqbit, which #755 already removed. Verified: cargo test/clippy/fmt clean. Regenerated a dev SBOM locally and validated it with cyclonedx-cli validate --fail-on-errors (same check test-container.yml runs in CI) -- passes, with the new properties landing on the osm-diffs component as expected and cdx:cbom:version gone. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
brawer
added a commit
that referenced
this pull request
Aug 24, 2026
#755 removed the last code path (librqbit's internal, unconfigured HTTP client) that ever depended on an OS-provided certificate store -- the only reqwest client left in the codebase already bundles Mozilla's own CA root bundle at compile time via webpki-roots (main.rs's build_client()), same as it always has. That was never written down anywhere, though, and #754 (bundling ca-certificates.crt into the scratch image instead) showed it's easy to reach for "give the container an OS trust store" without realizing there's a deliberate, existing alternative already in place, closer to what browsers like Firefox do. Adds a dedicated "Certificate trust" section to docs/SUPPLY_CHAIN_SECURITY.md explaining this -- placed right after the Containers/Minimal containers/Multi-architecture containers sequence (rather than wedged inside "Minimal containers" itself, which would've interrupted that build-topic flow) and right before Bill of Materials/SBOM and CBOM, so it flows straight into where the new CBOM properties (below) live -- plus a short pointer line from docs/SECURITY.md, matching that file's existing pattern of one-line routes to more detailed docs, rather than duplicating the explanation there. Also declares it as an explicit, queryable fact in the CBOM (scripts/sbom/pipeline.jq), extending the existing crypto:tls:* custom-property convention with crypto:catrust:source and crypto:catrust:osTrustStoreUsed on the osm-diffs component -- the webpki-roots crate itself already shows up as an ordinary SBOM library component (cargo-cyclonedx walks Cargo.lock), but that's just one more dependency among ~390 others, easy to miss; this makes "what CA trust store does this software use" answerable directly instead of inferred from a dependency's free-text description. Not modeled as a CycloneDX "certificate" cryptographic-asset: that type's cryptoProperties (subjectName, issuerName, notValidBefore/After, ...) describe a single certificate, not a curated bundle of ~140 unrelated root CAs. Also drops the pre-existing cdx:cbom:version property while in here: checked CycloneDX's actual property taxonomy (github.com/CycloneDX/cyclonedx-property-taxonomy) -- the cdx: namespace is reserved for officially registered CycloneDX properties only ("cbom" isn't among them: registered cdx sub-namespaces are ai-ml, composer, device, esbuild, gomod, lifecycle, maven, npm, pipenv, poetry, python, rustc, plus the standalone cdx:reproducible). Worse, there's nothing for a "CBOM version" to mean in the first place: grepped the actual CycloneDX 1.6 JSON schema (github.com/CycloneDX/specification) and "cbom" appears zero times in it. CBOM isn't a distinct document type or section with its own version -- it's just a CycloneDX document containing one or more cryptographic-asset components (which this file already has, e.g. crypto/protocol/tls-1.3). That property was a fossil from before IBM Research's original, standalone CBOM spec (github.com/IBM/CBOM, which did have its own version number) got absorbed into CycloneDX 1.6 as native cryptographic-asset support. Also drops a stale comment in build_client() referencing librqbit, which #755 already removed. Verified: cargo test/clippy/fmt clean. Regenerated a dev SBOM locally and validated it with cyclonedx-cli validate --fail-on-errors (same check test-container.yml runs in CI) -- passes, with the new properties landing on the osm-diffs component as expected and cdx:cbom:version gone. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened
The first real full-planet smoke test crashed
import_osmoutright, seconds in:import_atp(worldwide AllThePlaces fetch, real HTTPS) succeeded cleanly just before it, in the same process.Root cause
librqbit's internal HTTP(S) client (used to fetch the planet.torrentfile, and itshttps://webseed mirrors -- confirmed: the real torrent lists a dozen HTTPS mirrors alongside its trackers) builds its own, unconfiguredreqwest::Client-- confirmed inlibrqbit's own source, there's no hook to inject a preconfigured one. That falls through toreqwest's actual default in this version (0.13.4):rustls-platform-verifier, which on generic Linux callsrustls_native_certs::load_native_certs()to read the OS's system trust store. There is none in aFROM scratchimage.Our own
main.rs::build_client()-- used for the ATP fetch, which is why that step worked fine -- sidesteps this entirely by explicitly bundlingwebpki_roots::TLS_SERVER_ROOTSand calling.use_preconfigured_tls(). Both clients use the samerustlsstack (confirmed via Cargo's unified feature resolution) -- the difference is purely in where each one gets its root certificates from.Fix
apk add ca-certificatesin the Alpine builder stage, thenCOPYthe resulting bundle into the finalscratchimage at the same path. Confirmed empirically (ran the actual builder base image) that this produces a real, complete 3071-line Mozilla-style bundle at/etc/ssl/certs/ca-certificates.crt-- one of the standard pathsrustls_native_certschecks on Linux.Testing
Rebuilt the container locally (
podman build) and ran it against a real workdir. It got well past the exact point that failed before:import_atpcompleted real HTTPS transfers, hundreds of MB, no CA error anywhere. Never reachedimport_osmin the local test window, but only because this sandboxed dev machine's own network path was too slow for the full AllThePlaces dataset (real Hetzner hardware fetches it in ~3.5 minutes; this took over an hour and was still climbing) -- an environment limitation, not a signal about the fix itself. Full end-to-end confirmation deferred to the next real Hetzner full-planet run.🤖 Generated with Claude Code