Skip to content

perf(agent): isolate bulk traffic on a dedicated console - #1360

Open
appcypher wants to merge 2 commits into
appcypher/agent-raw-bulkfrom
appcypher/agent-bulk-console
Open

perf(agent): isolate bulk traffic on a dedicated console#1360
appcypher wants to merge 2 commits into
appcypher/agent-raw-bulkfrom
appcypher/agent-bulk-console

Conversation

@appcypher

@appcypher appcypher commented Aug 16, 2026

Copy link
Copy Markdown
Member

TL;DR

Put generation-7 raw records on a dedicated internal virtio-console lane, fix the measured bottlenecks, and preserve one public agent.sock.

Description

  • Keep control, cancellation, lifecycle, credit, and terminal messages on the original console; carry raw bulk on the internal agent-bulk console.
  • Make automatic selection the internal default: new runtime/new agentd uses dual port, while an old peer falls back to combined mode before operations begin.
  • Keep the transport selector hidden as a diagnostic and rollback control rather than exposing it through the public SDK or sandbox configuration.
  • Raise filesystem records to 3 MiB, coalesce bounded guest work, prioritize control, and let single active flows use the scheduler burst.
  • Replace the two-frame SDK output queue with byte-bounded delivery so healthy readers survive short bursts.
  • Preserve correlation-range quarantine and cleanup cuts across both lanes, and fix the full-duplex SSH/TCP backpressure cycle.
  • Restore buffered atomic copy_to_host publication as the default; callers can still explicitly request a file sync:
sandbox
    .fs()
    .copy_to_host_with_options(
        "/guest/data",
        "/host/data",
        HostCopyOptions {
            durability: HostCopyDurability::File,
        },
    )
    .await?;

The same fresh exact-head comparison is shown on every PR in this stack. “After” means all four PRs together at 7f3f44ce, with automatic dual-port selection and local-shm-v1 enabled; the original baseline is 05f53e7a. Both profiles ran twice in ABBA order.

Workload Original baseline Complete stack Change
FS host to guest, 256 MiB 153.5 MiB/s 323.4 MiB/s +110.6%
TCP host to guest, 64 MiB 198.8 MiB/s 210.1 MiB/s +5.7%
FS guest to host, 256 MiB 254.6 MiB/s 564.9 MiB/s +121.9%
TCP guest to host, 64 MiB 511.8 MiB/s 649.7 MiB/s +26.9%
TCP full duplex, 64 MiB each way 244.4 MiB/s 381.8 MiB/s +56.2%
Idle ping median 0.1383 ms 0.1471 ms +0.0088 ms
Ping under FS load, p95 480.3 ms 1.929 ms 99.60% lower

Environment: OVH Advance-4, Linux/KVM, 1-vCPU guest, host CPU 8 affinity, guest tmpfs, public Rust SDK, verified payload hashes, and empty stderr. After artifacts were rebuilt from 7f3f44ce against libkrun ff087a1.

Test Plan

  • cargo fmt --all -- --check
  • cargo check -p microsandbox-protocol -p microsandbox-agent-client -p microsandbox-runtime -p microsandbox
  • cargo clippy -p microsandbox-protocol -p microsandbox-agent-client -p microsandbox-runtime -p microsandbox --lib -- -D warnings
  • cargo test -p microsandbox-protocol -p microsandbox-agent-client -p microsandbox-runtime -p microsandbox --lib — 823 passed, 3 ignored
  • Mixed new/old peer tests prove combined-mode fallback
  • Run the same baseline-versus-complete-stack Linux transport benchmark shown across all four PRs

Add a bounded second virtio-console path for generation-7 raw records
while keeping control messages on the existing agent port.

Bind both ports to one boot, lease correlation ranges to random client
incarnations, and quarantine disconnected ranges until agentd confirms
owner-scoped cleanup.

Use bounded fair schedulers, cross-lane merge state, cancellation
tombstones, and transport-health propagation so bulk stalls cannot block
control traffic or corrupt a recycled operation.
@appcypher
appcypher force-pushed the appcypher/agent-bulk-console branch from f8d3530 to 37fb8d1 Compare August 16, 2026 09:34
Raise filesystem bulk records to 3 MiB and fuse bounded guest turns so one-vCPU transfers do less scheduling and copying work.

Make guest output routing byte-bounded without disconnecting healthy clients on short bursts, prioritize the control lane, preserve cleanup cuts, and bypass cross-lane actors in combined mode.

Break the in-process SSH duplex backpressure cycle and restore buffered host-copy publication as the historical default while retaining explicit file durability.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant