diff --git a/generated/benchmarks/EMBEDDING-BENCHMARKS.md b/generated/benchmarks/EMBEDDING-BENCHMARKS.md index 38181bfc..dee49723 100644 --- a/generated/benchmarks/EMBEDDING-BENCHMARKS.md +++ b/generated/benchmarks/EMBEDDING-BENCHMARKS.md @@ -6,6 +6,11 @@ Hit@N = expected symbol found in top N results. | Version | Model | Symbols | Hit@1 | Hit@3 | Hit@5 | Misses | Embed Time | |---------|-------|--------:|------:|------:|------:|-------:|-----------:| +| 3.10.0 | minilm | 1500 | 65.9% ↓1.5pp | 86.3% ↓0.9pp | 91.8% ↓0.6pp | 58 | 128.0s | +| 3.10.0 | jina-small | 1500 | 77.1% ↑1.3pp | 92.8% ↓0.8pp | 95.8% ↓0.7pp | 29 | 262.7s | +| 3.10.0 | jina-base | 1500 | 71.4% ↓0.7pp | 91.3% ↑1.0pp | 95.4% ~ | 35 | 1346.3s | +| 3.10.0 | nomic | 1500 | 81.3% ↑1.2pp | 94.7% ↓0.9pp | 97.5% ~ | 18 | 1340.3s | +| 3.10.0 | nomic-v1.5 | 1500 | 79.3% ↑0.9pp | 94.0% ↓0.9pp | 96.7% ~ | 24 | 1335.2s | | 3.9.6 | minilm | 1500 | 67.4% | 87.2% | 92.4% | 49 | 125.9s | | 3.9.6 | jina-small | 1500 | 75.8% | 93.6% | 96.5% | 23 | 253.8s | | 3.9.6 | jina-base | 1500 | 72.1% | 90.3% | 94.9% | 30 | 1327.4s | @@ -14,18 +19,86 @@ Hit@N = expected symbol found in top N results. ### Latest results -**Version:** 3.9.6 | **Strategy:** structured | **Symbols:** 1500 | **Date:** 2026-04-30 +**Version:** 3.10.0 | **Strategy:** structured | **Symbols:** 1500 | **Date:** 2026-05-11 | Model | Dim | Context | Hit@1 | Hit@3 | Hit@5 | Hit@10 | Misses | Embed | Search | |-------|----:|--------:|------:|------:|------:|-------:|-------:|------:|-------:| -| minilm | 384 | 256 | 67.4% | 87.2% | 92.4% | 96.7% | 49 | 125.9s | 130.5s | -| jina-small | 512 | 8192 | 75.8% | 93.6% | 96.5% | 98.5% | 23 | 253.8s | 159.2s | -| jina-base | 768 | 8192 | 72.1% | 90.3% | 94.9% | 98.0% | 30 | 1327.4s | 206.2s | -| nomic | 768 | 8192 | 80.1% | 95.5% | 97.9% | 99.0% | 15 | 1331.2s | 206.0s | -| nomic-v1.5 | 768 | 8192 | 78.4% | 94.9% | 97.1% | 98.8% | 18 | 1325.3s | 207.0s | +| minilm | 384 | 256 | 65.9% | 86.3% | 91.8% | 96.1% | 58 | 128.0s | 133.9s | +| jina-small | 512 | 8192 | 77.1% | 92.8% | 95.8% | 98.1% | 29 | 262.7s | 161.9s | +| jina-base | 768 | 8192 | 71.4% | 91.3% | 95.4% | 97.7% | 35 | 1346.3s | 209.5s | +| nomic | 768 | 8192 | 81.3% | 94.7% | 97.5% | 98.8% | 18 | 1340.3s | 210.5s | +| nomic-v1.5 | 768 | 8192 | 79.3% | 94.0% | 96.7% | 98.4% | 24 | 1335.2s | 208.1s |