Skip to content

PHASE_1_LAB.md

Latest commit

 

History

History
236 lines (179 loc) · 8.37 KB

PHASE_1_LAB.md

File metadata and controls

236 lines (179 loc) · 8.37 KB
Raw

Phase 1: Reproducible WAN Measurement Laboratory

Goal and current scope

Phase 1 creates a controlled path between a benchmark client and server, applies directional impairments, transfers deterministic data, verifies every byte, and records machine-readable results.

It answers: “How does direct TCP behave on this declared laboratory path?” It does not yet test a relay, QUIC, SOCKS5, or any acceleration feature.

Topology

na-client namespace       na-router namespace        na-server namespace
10.10.1.2/24              10.10.1.1   10.10.2.1      10.10.2.2/24
     na-c0 <------------> na-r0           na-r1 <------------> na-s0
                              |             |
                       reverse qdisc   forward qdisc
                      server -> client client -> server

The router namespace forwards IPv4 traffic. Queue disciplines are placed on its egress interfaces:

  • na-r1 controls client-to-server traffic;
  • na-r0 controls server-to-client traffic.

This arrangement makes asymmetric rate, delay, and loss explicit. It also avoids confusing one-way delay with RTT.

Prerequisites

  • A modern Linux host with network namespace support.
  • C++20 compiler and CMake 3.20 or newer.
  • Bash, Python 3, ip, tc, jq, sha256sum, and runuser.
  • Root access for lab setup and namespace execution.

Check tools:

command -v cmake g++ ip tc jq python3 runuser

Build:

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

Safe lifecycle

The topology uses names prefixed with na- and private laboratory subnets 10.10.1.0/24 and 10.10.2.0/24. Check that these names and routes do not conflict with an existing host setup.

sudo ./scripts/wan-lab.sh setup
./scripts/wan-lab.sh validate profiles/terrestrial-wan.env
sudo ./scripts/wan-lab.sh status
sudo ./scripts/wan-lab.sh apply profiles/terrestrial-wan.env
sudo ./scripts/wan-lab.sh clear
sudo ./scripts/wan-lab.sh destroy

setup, clear, and destroy are idempotent. apply replaces only the root queue disciplines on the two lab-router veth interfaces. It does not modify the host's physical interfaces or default route.

validate needs no privilege and prints the exact directional tc commands derived from a profile without changing network state.

If a run is interrupted, run clear or destroy. The namespaces do not provide Internet access and require none for the included benchmark.

Profiles

Profile Approximate behavior Purpose
clean.env No artificial impairment Namespace/tool overhead
terrestrial-wan.env 80 ms RTT, 100 Mbit/s Typical distant WAN control
leo-variable.env 80 ms RTT, jitter, asymmetry and light loss Variable lower-orbit-like lab case
geo.env 600 ms RTT, 25/5 Mbit/s High-BDP asymmetric path
geo-lossy.env GEO settings plus correlated random loss TCP recovery
burst-loss.env Gilbert-Elliott loss model Bursty failure behavior
reordered.env Controlled packet reordering Robustness and recovery

These are illustrative test fixtures, not claims about a specific satellite provider. Profile files accept only the allow-listed KEY=VALUE format; shell expressions are not evaluated.

Delay is configured per direction. For example, 300 ms forward plus 300 ms reverse produces approximately 600 ms base RTT. Actual observed latency also includes jitter, serialization, queueing, and processing.

The gemodel values follow tc-netem's Gilbert-Elliott parameters. Treat them as experiment inputs and retain them with results; do not translate them casually into a single “loss percent.”

Run a baseline

Create the topology once:

sudo ./scripts/wan-lab.sh setup

Run ten verified 64 MiB transfers:

sudo ./scripts/run-baseline.sh profiles/clean.env 10 67108864
sudo ./scripts/run-baseline.sh profiles/terrestrial-wan.env 10 67108864
sudo ./scripts/run-baseline.sh profiles/geo.env 10 67108864

The script:

  1. validates and applies the chosen profile;
  2. captures kernel, compiler, Git commit, profile hash, exact pre-run tc status, run count, byte count, and UTC time;
  3. starts the receiver in na-server;
  4. runs the sender in na-client;
  5. appends one JSON object per run; and
  6. rejects any result whose receiver verification failed; and
  7. records post-run tc counters in the metadata.

The default deterministic xorshift payload is intentionally difficult to compress. FNV-1a is used as a fast accidental-corruption check in addition to byte-for-byte comparison at the receiver; it is not a cryptographic integrity mechanism.

Keep the server run count equal to the client run count. The client uses a distinct seed for each run.

Result format

Raw files use newline-delimited JSON (NDJSON), one independent object per run:

{
  "schema_version": 1,
  "timestamp_utc": "2026-07-03T08:00:00Z",
  "profile": "terrestrial-wan",
  "method": "direct_tcp",
  "host": "10.10.2.2",
  "port": 5201,
  "run": 1,
  "seed": 1,
  "bytes": 67108864,
  "duration_seconds": 8.123456,
  "goodput_mbps": 66.089123,
  "hash_fnv1a64": "0123456789abcdef",
  "verified": true
}

Goodput is verified application payload divided by elapsed client time. It is not socket write speed and excludes protocol headers.

The companion metadata JSON records environment and qdisc state. Raw local files are Git-ignored. Commit evidence used for a design decision in a reviewed subdirectory with its raw runs, metadata, summary, and written conclusion.

Summarize and plot

The summarizer uses only the Python standard library:

python3 scripts/summarize-results.py \
  results/*-direct.ndjson \
  --csv results/summary.csv \
  --svg results/goodput.svg

It rejects unsupported schemas and unverified records, groups by profile and method, calculates median and p05/p95 goodput, and creates a dependency-free SVG chart. With fewer than ten runs, percentiles are descriptive only.

Manual diagnostics

sudo ./scripts/wan-lab.sh status
sudo ./scripts/wan-lab.sh exec client ping -c 5 10.10.2.2
sudo ./scripts/wan-lab.sh exec client ss -tin
sudo ip netns exec na-router tc -s qdisc show dev na-r1
sudo ip netns exec na-router tc -s qdisc show dev na-r0

Check measured RTT against the sum of directional delays. Check tc -s counters for sent, dropped, and requeued packets. A rate cap is not guaranteed to be perfect at short time scales; use sufficiently long runs.

Experiment rules

  • Change one independent variable at a time.
  • Run a clean-profile warm-up before measured runs.
  • Use at least ten measured repetitions for an initial comparison.
  • Randomize method order after additional transports exist.
  • Keep byte count, seeds, host load, build type, and profile fixed.
  • Record failures; never delete an inconvenient run without an explanation.
  • Compare medians and distributions, not only the best result.
  • Monitor CPU and memory before attributing an improvement to networking.
  • Call these “emulated profiles,” not real satellite measurements.

Known limitations and next work

The Phase 1 benchmark currently measures bulk direct-TCP goodput. Small sequential requests, concurrent flows, mixed interactive/bulk traffic, temporary outages, CPU/RSS capture, and orchestration across two physical hosts remain extensions to the measurement harness. The profile set provides jitter, loss, reordering, and asymmetry, but netem cannot reproduce every radio scheduler or real cross traffic.

Phase 2 may start after the clean and impaired baselines are executed on the intended development host and retained. The same benchmark and result schema will then compare direct TCP with the plain paired relay.

Troubleshooting

Operation not permitted — namespace and qdisc operations require root and may be blocked inside an unprivileged container.

File exists during setup — run destroy, verify that no unrelated na-* interfaces use these names, and retry.

Client timeout on GEO profile — increase --timeout; the default is 30 seconds and includes transfer plus acknowledgement time.

Very low clean-profile throughput — check host CPU contention, debug/sanitizer builds, veth offload settings, and payload size before changing the benchmark.

Results differ between hosts — compare metadata, kernel, compiler, CPU, virtualization, qdisc output, and offload configuration. That difference is data, not necessarily a bug.