Adaptive concurrency: RTT predictor emits degenerate predictions from near-singular fits

[assafvayner]

The adaptive concurrency RTT predictor (xet_client/src/cas_client/adaptive_concurrency/) can emit physically-impossible predictions when its weighted linear regression is near-singular — too few effective samples, or near-uniform transfer sizes at a single concurrency level. Filing to confirm whether this is a real controller bug: do these values feed concurrency decisions, or only reporting/logging?

Mechanism

• predicted_bandwidth() ≈ 9.5 TiB/s is the 1e-6 floor, not a measurement. When predicted_rtt(10 MiB, conc=1) comes out negative it is clamped to 0 (rtt_prediction.rs:94), then 10 MiB / min_rtt.max(1e-6) gives 10·1024·1024 / 1e-6 = 9.5367 TiB/s (rtt_prediction.rs:115-121) — a data-independent constant.
• predicted_max_rtt reaches tens of seconds: the spurious regression slope extrapolated out to the 64 MiB reference query point.
• The degenerate-fit signal is dropped. df = sw − 2 ≤ 0 returns (Some(mean), None) (exp_weighted_olr.rs:86-87), and the controller forwards the mean anyway via .unwrap_or(0.) (controller.rs:143-145).
• The singularity guard is absolute (delta.abs() < 1e-12, exp_weighted_olr.rs:75), so a tiny-but-nonzero delta (near-collinear x) passes and yields an exploding slope / large-negative intercept.

Trigger

Low x-variance regime: early in a transfer, or uniform-size transfers at one concurrency level → the regression x-axis has almost no spread, so RTT jitter drives a near-vertical slope that is then extrapolated far past the observed size range.

Suggested direction

Don't emit predictions outside the identifiable regime:

• Gate predict() on a relative condition number / minimum x-spread (not the absolute 1e-12); return None for the mean when df ≤ 0.
• Have predicted_bandwidth() return None when the underlying RTT was clamped to 0, instead of dividing by the 1e-6 floor.
• Optionally cap extrapolation to near the observed size range.