Skip to content

feat(sdk)!: redesign runtime bootstrap - #1454

Open
appcypher wants to merge 7 commits into
releases/v0.7.0from
appcypher/sdk-runtime-bootstrap
Open

feat(sdk)!: redesign runtime bootstrap#1454
appcypher wants to merge 7 commits into
releases/v0.7.0from
appcypher/sdk-runtime-bootstrap

Conversation

@appcypher

@appcypher appcypher commented Aug 24, 2026

Copy link
Copy Markdown
Member

TL;DR

Redesign runtime distribution around explicit download-binaries and embed-binaries features. Resolve and install msb plus libkrunfw as one matched pair, while Agentd can be embedded at build time or supplied through MSB_AGENTD_PATH or global paths.agentd configuration at runtime.

Description

  • Replace the prebuilt feature with separate acquisition and embedding controls across the Rust SDK, filesystem crate, runtime crate, CLI, workflows, and development recipes.
  • Replace the legacy Rust setup::install and single-path resolvers with paired resolve_runtime, install_runtime, and ensure_runtime APIs that are read-only during resolution and fail closed on partial installations.
  • Install downloaded or embedded runtime archives into the normal MSB_HOME/{bin,lib} layout, with locking, staged publication, optional digest verification, and no implicit runtime download during ordinary SDK sandbox creation.
  • Add build-time runtime bundle inputs through MSB_EMBED_ARTIFACTS_DIR and MSB_EMBED_RUNTIME_BUNDLE_PATH.
  • Add eager, validated Agentd selection through MSB_AGENTD_PATH and global paths.agentd, with the embedded payload as the default fallback when embed-binaries is enabled.
  • Update the CLI wrapper, self-update flow, TypeScript, Python, and Ruby native bindings, documentation, and API compatibility tests for the new setup contract.
use microsandbox::{
    config::LocalConfig,
    setup::{InstallOptions, ensure_runtime},
};

let runtime = ensure_runtime(&LocalConfig::default(), InstallOptions::default()).await?;
println!("msb: {}", runtime.msb_path.display());
println!("libkrunfw: {}", runtime.libkrunfw_path.display());

Test Plan

  • cargo fmt --all -- --check
  • MSB_AGENTD_PATH="$PWD/build/agentd" cargo test -p microsandbox-filesystem --no-default-features (621 passed)
  • MSB_AGENTD_PATH="$PWD/build/agentd" cargo test -p microsandbox --lib --no-default-features --features keyring,net (617 passed, 3 ignored)
  • MSB_EMBED_ARTIFACTS_DIR="$PWD/build" cargo test -p microsandbox --lib --no-default-features --features embed-binaries,keyring,net setup::runtime
  • MSB_HOME=/tmp/msb-api-surface-home MSB_EMBED_ARTIFACTS_DIR="$PWD/build" cargo test -p microsandbox --test api_compat
  • MSB_HOME=/tmp/msb-api-surface-home MSB_EMBED_ARTIFACTS_DIR="$PWD/build" cargo test -p microsandbox --lib setup::runtime::tests --features embed-binaries
  • cargo check -p microsandbox --no-default-features --bin microsandbox
  • Run focused clippy and compile checks for the CLI, filesystem, runtime, Rust SDK, Node native binding, and Python native binding; run npm run typecheck in sdk/node-ts
  • Run the Linux, Windows ARM64/x64, and Ruby release lanes (runs in CI; the previous revision passed)

@appcypher
appcypher requested a review from toksdotdev as a code owner August 24, 2026 14:30
@mintlify

mintlify Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
microsandbox-staging 馃煝 Ready View Preview Aug 24, 2026, 2:32 PM

馃挕 Tip: Enable Workflows to automatically generate PRs for you.

@greptile-apps

greptile-apps Bot commented Aug 24, 2026

Copy link
Copy Markdown

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SDK or CLI requests local runtime] --> B[Resolve explicit environment or SDK package pair]
    B -->|Complete pair| C[Use matched msb and libkrunfw]
    B -->|No explicit pair| D[Check configured MSB_HOME]
    D -->|Complete pair| C
    D -->|Partial pair| E[Fail closed]
    D -->|Absent and explicit ensure/install| F[Select release, directory, archive, or embedded source]
    F --> G[Verify optional archive digest]
    G --> H[Acquire installation lock]
    H --> I[Stage both runtime files]
    I --> J[Publish libkrunfw then msb completion marker]
    J --> C
    K[MSB_AGENTD_PATH] --> L[Read and validate Agentd ELF]
    M[Embedded Agentd] --> L
    L --> N[Cache payload for guest bootstrap]
Loading

Reviews (1): Last reviewed commit: "feat(sdk)!: redesign runtime bootstrap" | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SDK or CLI] --> B{Resolve runtime pair}
    B -->|Complete configured install| C[Use msb and libkrunfw]
    B -->|Missing and installation requested| D{Runtime source}
    D -->|Release download| E[Download archive]
    D -->|Embedded archive| F[Read embedded bundle]
    D -->|Local archive or directory| G[Read local artifacts]
    E --> H[Validate optional digest]
    F --> I[Stage matched pair]
    G --> I
    H --> I
    I --> J[Publish under MSB_HOME bin and lib]
    J --> C
    C --> K{Select Agentd}
    K -->|MSB_AGENTD_PATH| L[External payload]
    K -->|paths.agentd| L
    K -->|Embedded fallback| M[Embedded payload]
    L --> N[Validate Linux ELF]
    M --> N
    N --> O[Spawn local sandbox runtime]
Loading

Reviews (2): Last reviewed commit: "feat(sdk): support configured agentd pat..." | Re-trigger Greptile

@appcypher
appcypher changed the base branch from main to releases/v0.7.0 August 25, 2026 20:41
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SDK or CLI startup] --> B{Agentd override configured?}
    B -->|Yes| C[Read and validate MSB_AGENTD_PATH]
    B -->|No| D{Embedded Agentd available?}
    D -->|Yes| E[Use embedded Agentd payload]
    D -->|No| F[Fail before VM construction]
    C --> G[Cache selected Agentd payload]
    E --> G
    A --> H[Resolve matched msb and libkrunfw pair]
    H --> I{Complete valid installation?}
    I -->|Yes| J[Start local runtime]
    I -->|No and ensure requested| K[Download or unpack staged runtime bundle]
    K --> L[Validate and publish bin/lib files]
    L --> J
    I -->|No and resolution only| M[Return setup error]
    G --> J
Loading

Reviews (3): Last reviewed commit: "fix(filesystem): validate embedded agent..." | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[SDK or CLI] --> B{Runtime operation}
  B -->|Resolve| C[Check configured and installed paths]
  B -->|Ensure or install| D{Artifact source}
  D --> E[Embedded runtime bundle]
  D --> F[Downloaded runtime bundle]
  E --> G[Install matched msb and libkrunfw]
  F --> G
  G --> H[MSB_HOME bin and lib]
  A --> I{Agentd selection}
  I --> J[MSB_AGENTD_PATH]
  I --> K[Embedded Agentd fallback]
  H --> L[Local sandbox runtime]
  J --> L
  K --> L
Loading

Reviews (4): Last reviewed commit: "refactor(sdk): simplify runtime ensure o..." | Re-trigger Greptile

appcypher added a commit that referenced this pull request Aug 30, 2026
## TL;DR

Split the Rust SDK into explicit local and cloud feature surfaces, and
keep the existing all-capabilities default. A cloud-only SDK build now
avoids the local database, image, migration, metrics, VM runner, and
host-network-engine dependency trees.

This PR is stacked on #1454 (`appcypher/sdk-runtime-bootstrap`). After
#1454 merges, this PR should be retargeted to `releases/v0.7.0`.

## Description

- Add `local` and `cloud` SDK features while leaving both enabled by
default.
- Keep snapshots and image-archive support inside `local`, as requested,
instead of exposing product-level features for them.
- Make `download-binaries` and `embed-binaries` imply `local`; keep
Agentd packaging owned by the `msb` build.
- Split `microsandbox-runtime` into physical `client/` and `runner/`
trees, preserving its existing default behavior and public paths while
letting the SDK compile only the lightweight client contract.
- Split `microsandbox-network` into physical `model/` and `engine/`
trees, preserving its existing default behavior and public paths while
letting the SDK compile only configuration and policy types.
- Move `RegistryAuth` and the shared log-level contract into
`microsandbox-types` while preserving existing public re-export names.
- Remove unused direct dependencies from the SDK, runtime, image,
network, metrics, and utils crates.
- Document the supported feature combinations and add CI checks for
them.

The intentional compatibility boundary is limited to internal crates
built with `default-features = false`: callers that need the VM
implementation must now enable runtime `runner`, and callers that need
the host networking implementation must enable network `engine`.
Published defaults and the SDK's default behavior remain unchanged.

## Build Performance

| Metric | Before (#1454) | After (#1479) | Difference | Improvement |
| --- | ---: | ---: | ---: | ---: |
| Debug build | 66.22s | 44.98s | -21.24s | 32.1% faster |
| Release build | 99.40s | 78.37s | -21.03s | 21.2% faster |
| Dependencies | 526 | 432 | -94 | 17.9% fewer |
| Debug build directory | 2,495,848 KiB | 1,952,260 KiB | -543,588 KiB |
21.8% smaller |
| Release build directory | 1,150,812 KiB | 973,772 KiB | -177,040 KiB |
15.4% smaller |

Measured on an Apple Silicon Mac using clean target directories,
`CARGO_INCREMENTAL=0`, `--locked`, the same local runtime artifacts, and
a warm Cargo source cache. Each timing is one clean SDK library build.
The cloud-only surface has 230 dependencies and built in 27.60s debug
and 44.45s release under the same conditions.

## Test Plan

- [x] `cargo fmt --all -- --check`
- [x] `cargo clippy --workspace --exclude microsandbox-agentd -- -D
warnings`
- [x] `RUSTDOCFLAGS='-D warnings' cargo doc --workspace --exclude
microsandbox-agentd --no-deps`
- [x] `cargo check -p microsandbox --no-default-features --features
cloud,net`
- [x] `cargo check -p microsandbox --no-default-features --features
local,net`
- [x] `cargo check -p microsandbox-runtime --no-default-features
--features client,net`
- [x] `cargo check -p microsandbox-network --no-default-features`
- [x] `cargo check -p microsandbox-cli --no-default-features`
- [x] `cargo test -p microsandbox-network --lib` (486 passed)
- [x] `cargo test -p microsandbox-runtime --lib` (137 passed)
- [x] `cargo test -p microsandbox --lib --no-default-features --features
cloud,keyring,local,net` (620 passed, 3 ignored)
- [x] `cargo check` in `sdk/rust/fuzz`
- [x] Real macOS microVM boot, guest command execution, stop, and
cleanup through the Rust SDK
@greptile-apps

greptile-apps Bot commented Aug 30, 2026

Copy link
Copy Markdown

Too many files changed for review (190 files, 100 file limit).

Bypass the limit by tagging @greptile-apps to review.

Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Comment thread sdk/rust/lib/setup/runtime.rs Fixed
Replace the prebuilt feature and single-path setup helpers with explicit download-binaries and embed-binaries behavior. Resolve, install, and verify msb plus libkrunfw as one matched pair, while allowing a validated runtime Agentd override.

Update the CLI, native SDK bindings, workflows, and documentation to use the paired setup API and fail closed on partial installations.
Allow the global configuration to select an external Agentd payload while preserving MSB_AGENTD_PATH as the highest-precedence override.

Forward the selected path to spawned msb processes and fail closed when an explicit payload is missing or invalid. Document the resolution order and cover it with focused tests.
Share Agentd ELF validation between build-time staging and runtime resolution so release builds reject malformed or architecture-mismatched guest payloads before producing artifacts.

Point Windows Quality at its downloaded target-matched Agentd when exercising non-embedded runtime tests.
Pass InstallOptions directly to ensure_runtime instead of wrapping it in a one-field EnsureOptions type. Update embedded and Cargo wrapper callers plus the API compatibility test for the smaller setup surface.

Clarify runtime and Agentd feature behavior in the setup documentation, and replace the stale Go resolver reference.
Split the Rust SDK into explicit local and cloud feature surfaces, and
keep the existing all-capabilities default. A cloud-only SDK build now
avoids the local database, image, migration, metrics, VM runner, and
host-network-engine dependency trees.

This PR is stacked on #1454 (`appcypher/sdk-runtime-bootstrap`). After

- Add `local` and `cloud` SDK features while leaving both enabled by
default.
- Keep snapshots and image-archive support inside `local`, as requested,
instead of exposing product-level features for them.
- Make `download-binaries` and `embed-binaries` imply `local`; keep
Agentd packaging owned by the `msb` build.
- Split `microsandbox-runtime` into physical `client/` and `runner/`
trees, preserving its existing default behavior and public paths while
letting the SDK compile only the lightweight client contract.
- Split `microsandbox-network` into physical `model/` and `engine/`
trees, preserving its existing default behavior and public paths while
letting the SDK compile only configuration and policy types.
- Move `RegistryAuth` and the shared log-level contract into
`microsandbox-types` while preserving existing public re-export names.
- Remove unused direct dependencies from the SDK, runtime, image,
network, metrics, and utils crates.
- Document the supported feature combinations and add CI checks for
them.

The intentional compatibility boundary is limited to internal crates
built with `default-features = false`: callers that need the VM
implementation must now enable runtime `runner`, and callers that need
the host networking implementation must enable network `engine`.
Published defaults and the SDK's default behavior remain unchanged.

| Metric | Before (#1454) | After (#1479) | Difference | Improvement |
| --- | ---: | ---: | ---: | ---: |
| Debug build | 66.22s | 44.98s | -21.24s | 32.1% faster |
| Release build | 99.40s | 78.37s | -21.03s | 21.2% faster |
| Dependencies | 526 | 432 | -94 | 17.9% fewer |
| Debug build directory | 2,495,848 KiB | 1,952,260 KiB | -543,588 KiB |
21.8% smaller |
| Release build directory | 1,150,812 KiB | 973,772 KiB | -177,040 KiB |
15.4% smaller |

Measured on an Apple Silicon Mac using clean target directories,
`CARGO_INCREMENTAL=0`, `--locked`, the same local runtime artifacts, and
a warm Cargo source cache. Each timing is one clean SDK library build.
The cloud-only surface has 230 dependencies and built in 27.60s debug
and 44.45s release under the same conditions.

- [x] `cargo fmt --all -- --check`
- [x] `cargo clippy --workspace --exclude microsandbox-agentd -- -D
warnings`
- [x] `RUSTDOCFLAGS='-D warnings' cargo doc --workspace --exclude
microsandbox-agentd --no-deps`
- [x] `cargo check -p microsandbox --no-default-features --features
cloud,net`
- [x] `cargo check -p microsandbox --no-default-features --features
local,net`
- [x] `cargo check -p microsandbox-runtime --no-default-features
--features client,net`
- [x] `cargo check -p microsandbox-network --no-default-features`
- [x] `cargo check -p microsandbox-cli --no-default-features`
- [x] `cargo test -p microsandbox-network --lib` (486 passed)
- [x] `cargo test -p microsandbox-runtime --lib` (137 passed)
- [x] `cargo test -p microsandbox --lib --no-default-features --features
cloud,keyring,local,net` (620 passed, 3 ignored)
- [x] `cargo check` in `sdk/rust/fuzz`
- [x] Real macOS microVM boot, guest command execution, stop, and
cleanup through the Rust SDK
Pin renamed napi object fields to their public TypeScript names and return concrete sandbox types so generated declarations contain no dangling Rust identifiers.

Type-check native/index.d.ts after every native build and from the standard typecheck command to prevent regressions during release packaging.
Keep backend-specific constructors, stream variants, and tests behind their owning feature gates so local-only and cloud-only builds remain valid.

Preserve Node release target forwarding through npm lifecycle hooks, enable the Ruby binding backend surface explicitly, and keep Rust integration-test TLS dependencies direct.

Rewrite runtime installation assertions around trusted expected paths so CodeQL does not mistake returned test paths for uncontrolled filesystem access.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants