feat: two-gem split — SDK-only microsandbox-rb + microsandbox-rb-binaries runtime gem - #58
Conversation
…le time Pull the core crate with default-features = false, features = ["keyring", "net", "ssh"]: the SDK crate's `prebuilt` feature made its build.rs fetch msb + libkrunfw into ~/.microsandbox while compiling the extension. The host runtime now comes from the companion microsandbox-rb-binaries gem or, as the lowest tier, the first-use download (setup::install is not gated on prebuilt). The guest agentd must still be embedded: microsandbox-filesystem's build.rs demands a locally built build/agentd without ITS prebuilt feature (the two downloads shared one umbrella feature upstream), so enable just that sub-feature through a direct microsandbox-runtime dependency. Cargo.lock loses only the SDK-level prebuilt's transitive deps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
One gem per platform (arm64-darwin, x86_64-linux-gnu, aarch64-linux-gnu)
shipping vendor/bin/msb + vendor/lib/libkrunfw.* + vendor/manifest.json from
the upstream release named by Microsandbox::Binaries::RUNTIME_VERSION.
`rake -C binaries vendor[<platform>]` downloads the release bundle and verifies
it fail-closed against the release's checksums.sha256 as committed in
binaries/checksums/<tag>.sha256 (the live file must agree — a GitHub release is
mutable); unknown bundle entries, missing digests, symlinks and partial trees
all abort. `build[<platform>]` re-verifies the staged tree against the manifest
and sets the payload explicitly (Gem::Specification.load caches by path, so the
gemspec's vendor/ glob cannot be trusted across a vendor:all run). `verify`
runs the vendored msb on the host. No executables, no dependency edge to the
SDK gem in either direction.
Gemfile: `gemspec glob: "{,*}.gemspec"` — Bundler's default glob evaluated the
nested gemspec in every `bundle exec` process.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
`require "microsandbox"` now activates the companion gem once, eagerly (like the Node SDK's platform package at module load, so every msb-spawning entry point sees it, not just Sandbox.create/start): lockstep `gem "…", "= VERSION"` pin → `require "microsandbox/binaries"` (absent → silent) → the provider must be the companion gem, this gem's own tree or a bare load path → RUNTIME_VERSION must match (else warn + skip — a stale msb passes an exists-check and then fails every create on a wire-protocol mismatch) → msb AND libkrunfw present → Native.set_runtime_msb_path. Never raises. Effective ladder: MSB_PATH → binaries gem → config → ~/.microsandbox → PATH. ensure_runtime! skips the ~/.microsandbox installer when the resolver returns the bundled msb; runtime_path= warns that the set-once slot is already claimed. Integration specs now gate on a resolvable runtime path rather than `installed?` (home-dir only), so a bundled runtime runs them instead of skipping all of them. Specs: in-process activation matrix, lockstep guard for the companion gem's VERSION/RUNTIME_VERSION, and subprocess specs against the real set-once slot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
…ment the split CI: a `binaries` job vendors + builds all three platform gems and smoke-runs the host one from the installed gem; `package` installs source gem + binaries gem with plain `gem install` and asserts the resolver lands in the gem, msb runs, and the installer is skipped; `integration` gains a `bundled` lane booting real microVMs from the gem's vendored runtime (RUBYLIB=binaries/lib, auto-install off) next to the `provisioned` fallback lane, and both assert that examples actually ran (rspec exits 0 when everything is skipped). Docs: README installation/runtime-configuration/versioning/releasing, DESIGN runtime section, CHANGELOG Unreleased, CLAUDE.md third lock-step axis. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 26f7baa259
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| end | ||
| Native.set_runtime_msb_path(msb) | ||
| @bundled_msb_path = msb | ||
| rescue => e |
There was a problem hiding this comment.
When the installed companion's microsandbox/binaries.rb is corrupt or uses syntax unsupported by the running Ruby, require raises SyntaxError, which inherits from ScriptError rather than StandardError; neither the inner rescue LoadError nor this rescue => e catches it. Consequently, the optional companion can make require "microsandbox" fail—even for cloud-only users—instead of following the documented warn-and-fallback behavior. Catch SyntaxError at the companion load boundary as well.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed: the activation boundary now rescues StandardError, ScriptError (with a comment on why), and a spec raises SyntaxError out of the stubbed require to pin it (spec/unit/runtime_spec.rb).
…ample Both lanes of the first dispatch booted real microVMs (49 examples, 0 failures) and were then failed by the new `pending_count == 0` assertion over attach_spec's legitimate "requires a real TTY" skip. Keep the guard against a lane that silently skips because its runtime never resolved — that is what the bundled lane exists to catch — by matching spec_helper's runtime-gate skip messages, and list any other pending examples for the log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 00522849a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "(only microsandbox-rb-binaries is trusted for the bundled runtime)" | ||
| return nil | ||
| end | ||
| unless Binaries::RUNTIME_VERSION == RUNTIME_VERSION |
There was a problem hiding this comment.
When Bundler has selected a different microsandbox-rb-binaries version, the exact-version gem call above raises Gem::LoadError, which is deliberately swallowed, and require then loads the mismatched bundled version. This condition only checks RUNTIME_VERSION, so gem-only SDK releases that share the same upstream runtime tag are accepted despite the documented lockstep requirement; an older companion API or packaging revision can therefore claim the resolver slot. Include Binaries::VERSION == VERSION in the activation gate rather than relying on the rescued activation attempt.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed — the gem "…", "= VERSION" pin is best-effort under Bundler, so it cannot be the enforcement. The gate now requires Binaries::VERSION == VERSION and Binaries::RUNTIME_VERSION == RUNTIME_VERSION; either mismatch warns and falls back to ~/.microsandbox. Spec added for the same-runtime/different-gem-version case; README/DESIGN/CHANGELOG/binaries README updated to say "same version and same runtime".
|
Merge-gate evidence — |
…ive SyntaxError Review (Codex) on #58: - The activation gate only compared RUNTIME_VERSION, so a companion gem at a different gem version but the same runtime tag was accepted although the documented contract is "install both at the same version" — and the `gem "…", "= VERSION"` pin cannot enforce it (under Bundler it raises for any other bundled version and is swallowed). Require Binaries::VERSION == VERSION as well; either mismatch warns and falls back. - A corrupt / newer-syntax binaries.rb raises SyntaxError out of `require`, which is a ScriptError, not a StandardError: neither rescue caught it, so an optional gem could take `require "microsandbox"` down — even for cloud-only users. Rescue ScriptError at the activation boundary too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH
1 participant
What
Clean-start implementation of the two-gem split agreed in upstream superradcompany/microsandbox#1305 — one thing only:
microsandbox-rbbecomes SDK-only, and a newmicrosandbox-rb-binariesgem carries the runtime. Supersedes the evidence-gathering prototype in #55 (closed in favour of this).microsandbox-rbis now SDK-only. The core crate is pulled withdefault-features = false, features = ["keyring", "net", "ssh"]: the SDK crate'sprebuiltfeature — whosebuild.rsdownloaded msb + libkrunfw into~/.microsandboxat compile time — is off. The guestagentdis still embedded: that comes frommicrosandbox-filesystem/prebuilt, kept via a directmicrosandbox-runtime = { features = ["prebuilt"] }dep (upstream bundles both downloads under one umbrella feature; without this the build fails on a missingbuild/agentd). Nothing is provisioned at gem build/install time anymore.microsandbox-rb-binaries(binaries/): one platform gem each forarm64-darwin,x86_64-linux-gnu,aarch64-linux-gnu, shippingvendor/bin/msb+vendor/lib/libkrunfw.*+vendor/manifest.json.rake -C binaries vendor[<platform>]downloads the upstream release bundle named byMicrosandbox::Binaries::RUNTIME_VERSIONand verifies it fail-closed against the release'schecksums.sha256as committed inbinaries/checksums/<tag>.sha256(reviewed in at runtime adoption; the live file must agree — a GitHub release is mutable). Unknown bundle entries, missing digests, symlinks and partial trees abort.build[<platform>]re-verifies the staged tree against the manifest and sets the payload explicitly from it (Gem::Specification.loadcaches by path, so the gemspec glob can't be trusted across avendor:allrun). No executables; no dependency edge in either direction (RubyGems has no optional deps; cloud-only users skip ~50 MB).require "microsandbox"tries the companion gem once, eagerly (like the Node SDK's platform package at module load, so every msb-spawning entry point sees it — a lazy hook inensure_runtime!would only coverSandbox.create/start): lockstepgem "…", "= VERSION"pin →require "microsandbox/binaries"→ the provider must be the companion gem, this gem's own tree, or a bare load path (a barerequirewould otherwise activate any gem shipping that feature path) →RUNTIME_VERSIONmust match (else warn + skip, never a stale-msb wire mismatch) → msb and libkrunfw present →Native.set_runtime_msb_path. Never raises. Effective ladder:MSB_PATH→ binaries gem → config →~/.microsandbox→PATH.ensure_runtime!skips the installer when the gem's msb is what resolves;runtime_path=warns that the set-once slot is already claimed (useMSB_PATH). The first-use download into~/.microsandboxstays as the lowest tier (npx parity, per #1305).spec/spec_helper.rb): examples now skip only whenMicrosandbox.runtime_pathdoes not resolve to a file, instead of oninstalled?— which checks~/.microsandboxonly, so with the runtime coming from the gem (orMSB_PATH) every example skipped and the suite went green having booted nothing.spec/unit/version_spec.rbassertsBinaries::VERSION == VERSIONandBinaries::RUNTIME_VERSION == RUNTIME_VERSION.binariesjob (vendor + build all three platform gems, smoke-run the host one from the installed gem, upload artifact);packagejob installs source gem + binaries gem with plaingem installand asserts the resolver lands in the gem, msb runs, installer is skipped;integrationnow runs aprovisionedlane (fallback tier) and abundledlane that boots real microVMs from the gem's vendored runtime (RUBYLIB=binaries/lib, auto-install disabled so a silently inactive tier can't be masked). Both lanes assert that examples actually ran and none were pending (rspec exits 0 when everything is skipped).gemspec glob: "{,*}.gemspec"— Bundler's default glob evaluated the nested gemspec in everybundle execprocess.Zero changes to the Rust extension sources; no CLI shim, no demo, no empty ruby-platform fallback gem (appcypher: moot post-split).
Not in this PR (follow-ups)
microsandbox-rb-binaries: needs a pending trusted publisher for the new gem name on rubygems.org first (OIDC, like the main gem), then release.yml wiring (build frombinaries/pkg/, push alongside the source gem). Until then CI builds the gems as artifacts only; the optionalgem "microsandbox-rb-binaries"simply isn't resolvable yet.~/.microsandboxfirst-use download still has no digest check (upstream #1300 pending; fix(deps): adopt v0.6.8 digest-verification backport for runtime downloads #54 tracked a backport). The binaries-gem path is verified at build time.Verification
-D warningsclean,cargo checkwithout the SDK-levelprebuilt.rake -C binaries vendor:allbuilt all three platform gems (23–27 MB each, correct per-platform payload);gem installed the arm64-darwin one into an isolatedGEM_HOME,ruby -I lib→Microsandbox.runtime_pathresolves into the installed gem,msb --version→msb 0.6.9,ensure_runtime!skips the installer; same underbundle execwithRUBYLIB(the integration bundled-lane mechanism).Gem::Specification.loadpath cache breakingvendor:all; foreign-gem activation via barerequire; mutable upstream checksums (now pinned in-repo); symlinks passing manifest verification; silentruntime_path=no-op.integrationruns only on push-to-main /workflow_dispatch, so the two lanes are dispatched manually on this branch — merge gate is the bundled lane going green (real VMs booted from the gem's binaries).🤖 Generated with Claude Code
https://claude.ai/code/session_01D9upBMYy2uZqz5gp21qgTH