Self-measured on every release using auto-generated queries from symbol names.
Each symbol's name is split into words (e.g. buildGraph → "build graph") and used as the search query.
Hit@N = expected symbol found in top N results.
| Version | Model | Symbols | Hit@1 | Hit@3 | Hit@5 | Misses | Embed Time |
|---|---|---|---|---|---|---|---|
| 3.10.0 | minilm | 1500 | 65.9% ↓1.5pp | 86.3% ↓0.9pp | 91.8% ↓0.6pp | 58 | 128.0s |
| 3.10.0 | jina-small | 1500 | 77.1% ↑1.3pp | 92.8% ↓0.8pp | 95.8% ↓0.7pp | 29 | 262.7s |
| 3.10.0 | jina-base | 1500 | 71.4% ↓0.7pp | 91.3% ↑1.0pp | 95.4% ~ | 35 | 1346.3s |
| 3.10.0 | nomic | 1500 | 81.3% ↑1.2pp | 94.7% ↓0.9pp | 97.5% ~ | 18 | 1340.3s |
| 3.10.0 | nomic-v1.5 | 1500 | 79.3% ↑0.9pp | 94.0% ↓0.9pp | 96.7% ~ | 24 | 1335.2s |
| 3.9.6 | minilm | 1500 | 67.4% | 87.2% | 92.4% | 49 | 125.9s |
| 3.9.6 | jina-small | 1500 | 75.8% | 93.6% | 96.5% | 23 | 253.8s |
| 3.9.6 | jina-base | 1500 | 72.1% | 90.3% | 94.9% | 30 | 1327.4s |
| 3.9.6 | nomic | 1500 | 80.1% | 95.5% | 97.9% | 15 | 1331.2s |
| 3.9.6 | nomic-v1.5 | 1500 | 78.4% | 94.9% | 97.1% | 18 | 1325.3s |
Version: 3.10.0 | Strategy: structured | Symbols: 1500 | Date: 2026-05-11
| Model | Dim | Context | Hit@1 | Hit@3 | Hit@5 | Hit@10 | Misses | Embed | Search |
|---|---|---|---|---|---|---|---|---|---|
| minilm | 384 | 256 | 65.9% | 86.3% | 91.8% | 96.1% | 58 | 128.0s | 133.9s |
| jina-small | 512 | 8192 | 77.1% | 92.8% | 95.8% | 98.1% | 29 | 262.7s | 161.9s |
| jina-base | 768 | 8192 | 71.4% | 91.3% | 95.4% | 97.7% | 35 | 1346.3s | 209.5s |
| nomic | 768 | 8192 | 81.3% | 94.7% | 97.5% | 98.8% | 18 | 1340.3s | 210.5s |
| nomic-v1.5 | 768 | 8192 | 79.3% | 94.0% | 96.7% | 98.4% | 24 | 1335.2s | 208.1s |