Skip to content

fix(wave): make incomparability explicit, closing the 0.0-sentinel class - #69

Merged
NickFlach merged 1 commit into
masterfrom
fix/wave-comparability
Aug 11, 2026
Merged

NickFlach merged 1 commit into
masterfrom
fix/wave-comparability

Conversation

@NickFlach

Copy link
Copy Markdown
Collaborator

Implements the delegated decision on #67. Do not merge — for review. Decision posted on the issue before any code was written.

The audit found more than the issue described

I filed #67 against one conflation. cosine_similarity actually collapsed four distinct undefined conditions onto the same 0.0 that means "orthogonal":

Condition Why it isn't orthogonality
Length mismatch Not comparable at all
Empty input No vector to compare
Zero magnitude A zero vector has no direction — the angle is undefined, not 90°
Non-finite magnitude NaN/±inf input, or squaring-and-summing overflowed f32

xi_repulsive_force had the same shape with worse polarity: its 0.0 means no repulsion, so incomparable signatures read as identical — failing toward "these are the same thing".

The sharpest consequence, now rejected

A zero-magnitude embedding scored 0.0 against every peer, which reads as orthogonal to everything — i.e. maximal differentiation. So a corpus containing one empty embedding didn't merely lose information, it reported inflated Xi. That's the same inversion class as #60, one layer down.

Checked variants, not changed signatures

The decision allowed Result "if blast radius allows like #65". In-crate it does — but the comparison to #65 doesn't hold, and I'd rather say so than quietly diverge. compute_differentiation_xi had zero production callers, so changing its signature cost nothing. cosine_similarity is the opposite: the crate's hottest numeric primitive, called from downstream inner loops that score whole candidate sets. Forcing ? there to handle what is a caller bug taxes every well-behaved caller.

So try_cosine_similarity / try_xi_repulsive_force are additive, and every in-crate production path moved to them. cosine_similarity no longer appears in non-test code in metrics.rs, with a comment marking its reappearance as the regression.

The two passes needed different policies — my first attempt got this wrong

compute_xi_signature legitimately returns the zero vector whenever the nonlinear commutator cancels, and [1, 0] is enough to trigger it. Propagating ZeroMagnitude from the Xi-signature pass therefore rejected perfectly ordinary axis-aligned corpora. Six tests caught it.

Final policy: raw-embedding comparison propagates ZeroMagnitude (an absent embedding is caller error); Xi-signature comparison tolerates it as 0.0 (a structureless signature is a real state). Both documented on xi_signature_similarity.

Deliberately not done — documented, not silently decided

  • No debug_assert! in the lossy helpers. It was floated for compute_differentiation_xi() turns mixed-dimension embeddings into xi=0 #60's legacy path. It would turn a silent wrong number into a panic in every downstream debug build — a behaviour change imposed on consumers who may have legitimately ragged data today, not a hardening.
  • A structureless Xi signature still scores 0.0 against peers, which reads as "maximally different in Xi" and nudges xi_variance up — the very inflation shape this PR removes for raw embeddings. Arguably two structureless signatures should score 1.0 (identically featureless). Changing it recalibrates Xi for every axis-aligned corpus, the same cost that kept compute_differentiation_xi() collapses two-vector differentiation to 0 #35 piecewise, so it's flagged in-code for an explicit decision rather than folded into a hardening pass. Worth a look if you want this closed properly.

Changes

Added: wave::VectorCompareError, wave::try_cosine_similarity, metrics::try_xi_repulsive_force, XiSignature::try_repulsive_force, XiError::Incomparable + From<VectorCompareError>.

Changed: compute_differentiation_xi rejects zero-magnitude and non-finite embeddings; XiError is now #[non_exhaustive] (it shipped in 0.5.0 one day ago, so no downstream match should exist yet); xi_diversity_boost's pass-through on incomparable signatures is now stated policy rather than an accident of 0.0 falling through both tier thresholds — numeric behaviour unchanged.

Additive and non-breaking for existing signatures. 0.5.0 → 0.6.0 for the new API and changed documented policy.

Gates

cargo test --all-features                                   103 + 10 + 6 passed, 0 failed  (was 94 + 10 + 6)
cargo test                                                  passed
cargo clippy --all-targets --all-features -- -D warnings    clean
cargo fmt --all --check                                     clean
cargo check --no-default-features                           clean
cargo check --no-default-features --features serde          clean

Anti-vacuous, verified by mutation: folding ZeroMagnitude back to Ok(0.0) fails three tests. And the suite asserts genuine orthogonality stays Ok(0.0) rather than becoming an error — so the checked form cannot pass by rejecting everything, which is the obvious way this kind of test goes vacuous.

🤖 Generated with Claude Code

The audit found more than #67 originally described. cosine_similarity
collapsed FOUR distinct undefined conditions onto the same 0.0 that means
"orthogonal": length mismatch, empty input, zero magnitude, and non-finite
magnitude. A zero vector has no direction and a NaN-bearing vector has no
meaningful magnitude — neither is orthogonal to anything, yet all four
arrived at the caller as a plausible measurement.

xi_repulsive_force had the same shape with worse polarity: its 0.0 means
NO repulsion, so incomparable signatures read as identical. It failed
toward "these are the same thing".

Sharpest consequence, now rejected: a zero-magnitude embedding scored 0.0
against every peer, which reads as orthogonal to everything, i.e. maximal
differentiation. A corpus containing an empty embedding did not merely
lose information — it reported INFLATED Xi.

Checked variants rather than changing the existing signatures. The
delegation allowed Result "if blast radius allows like #65", and in-crate
it does, but the comparison does not hold: compute_differentiation_xi had
zero production callers, whereas cosine_similarity is the crate's hottest
numeric primitive and downstream scores whole candidate sets with it in
inner loops. Forcing `?` there to handle what is a caller bug taxes every
well-behaved caller. So try_cosine_similarity / try_xi_repulsive_force are
additive, and every in-crate production path moved to them —
cosine_similarity is now absent from non-test code in metrics.rs, with a
comment saying its reappearance would be the regression.

Raw embeddings and Xi signatures needed DIFFERENT policies, which the
first pass got wrong. compute_xi_signature legitimately returns the zero
vector whenever the nonlinear commutator cancels — [1, 0] is enough — so
propagating ZeroMagnitude from the Xi pass rejected perfectly ordinary
axis-aligned corpora. Six tests caught it. Xi-signature comparison now
tolerates ZeroMagnitude as 0.0; raw-embedding comparison propagates it.

Deliberately not done, both documented rather than silently decided:
- No debug_assert! in the lossy helpers. It would turn a silent wrong
  number into a panic in every downstream debug build — a behaviour
  change imposed on consumers, not a hardening.
- A structureless Xi signature still scores 0.0 against peers, which
  reads as "maximally different in Xi" and nudges xi_variance up: the
  same inflation shape this closes for raw embeddings. Arguably it should
  be 1.0 (identically featureless). Changing it recalibrates Xi for every
  axis-aligned corpus, the same cost that kept #35 piecewise, so it is
  flagged on xi_signature_similarity for an explicit decision.

Tests 94 -> 103. The ZeroMagnitude distinction is confirmed anti-vacuous:
folding it back to Ok(0.0) fails three tests, and genuine orthogonality is
asserted to remain Ok(0.0) rather than an error, so the checked form
cannot pass by rejecting everything.

Additive and non-breaking; 0.5.0 -> 0.6.0 for the new API and changed
documented policy. XiError is now #[non_exhaustive] — it shipped in 0.5.0
one day ago, so no downstream match should exist yet.

Fixes #67

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@NickFlach
NickFlach merged commit ae01ff3 into master Aug 11, 2026
1 check passed
@NickFlach
NickFlach deleted the fix/wave-comparability branch August 11, 2026 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant