Skip to content

Commit a5c2eeb

Browse files
committed
Fix index freshness, ignore semantics, and search fallback consistency
1 parent 772a648 commit a5c2eeb

14 files changed

Lines changed: 313 additions & 140 deletions

‎README.md‎

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Remote LLM endpoints must use HTTPS; HTTP is accepted only on loopback.
5252

5353
**Synonyms:** Each node (file or directory) gets 5–12 LLM-generated synonyms via batched API calls to an OpenAI-compatible endpoint. For example, `Skeletons.tsx → ["skeletons", "loading", "placeholder", "animation"]`. Names are smart-batched (up to 60 per request, 10 parallel by default). The prompt includes extracted symbols and, for JS/TS files, ESM imports to generate contextual terms. Use `--offline` to guarantee that no LLM requests are made.
5454

55-
**Bootstrap diff:** On subsequent runs, `init` diffs the filesystem against the existing snapshot using file modification times. Deleted files are removed, new files are added, modified files are re-extracted. No LLM calls for existing synonyms — run `ctxt sync` to fill gaps. Synonym keys use `parentDir/basename` (e.g., `webhook/route.ts`) instead of bare basenames to prevent duplicate filenames from overwriting each other in the synonym map.
55+
**Bootstrap diff:** On subsequent runs, `watch` diffs the filesystem against the existing snapshot using file modification times and sizes. Timestamp-preserving edits with a different size are detected; edits preserving both size and timestamp require a full rebuild. Deleted files are removed, new files are added, modified files are re-extracted. No LLM calls for existing synonyms — run `ctxt sync` to fill gaps. Synonym keys use `parentDir/basename` (e.g., `webhook/route.ts`) instead of bare basenames to prevent duplicate filenames from overwriting each other in the synonym map.
5656

5757
### Search mechanics
5858

@@ -77,7 +77,7 @@ The `--explain` flag reveals this breakdown:
7777
basename contains +7: skeleton + syn exact +8: loading + sym contains +5: PageSkeleton = total 148
7878
```
7979

80-
Results are ranked by total score. Low-signal short/common words are filtered from query and synonym matching to reduce noise. Results are truncated at confidence gaps — if there's a 50%+ score drop between consecutive results, lower-scoring results are omitted even if `--limit` would include them.
80+
Results are ranked by total score. Low-signal short/common words are filtered from query and synonym matching to reduce noise. Results are truncated at confidence gaps — if there's a 50%+ score drop between consecutive results, lower-scoring results are omitted even if `--limit` would include them. Exact filename/symbol lookups can return just one strong match; conceptual queries retain up to three alternatives. Zero-score candidates are never returned.
8181

8282
### Watch mode
8383

@@ -145,7 +145,7 @@ Flags:
145145
- `--memory` (default true) — query live watch index first, fall back to snapshot
146146
- `--memory-only` — fail if live memory unavailable
147147
- `--runtime-file` — path to runtime discovery file (default `.ctxt/ctx_runtime.json`)
148-
- `--hybrid` — augment index results with content matching via ripgrep (default false)
148+
- `--hybrid` — augment index results by searching indexed files with ripgrep (default false); honors type and minimum-score filters
149149
- `--hybrid-score` — score assigned to content-matched results (default 1)
150150
- `--hybrid-root` — project root for content matching (defaults to index root)
151151

@@ -272,11 +272,12 @@ Contexting loads common `.gitignore` patterns by default. Additional ignores com
272272
2. **`.gitignore` patterns** — loaded from the project root
273273
3. **`ignore` in config** — extra patterns merged with defaults
274274

275-
Ignore sources are additive. Negated `.gitignore` rules (`!path`) are not
276-
supported, so a later rule cannot re-include an earlier ignored path. Symlink
277-
entries are not followed or indexed.
275+
Root `.gitignore` rules support ordered negation (`!path`), rooted patterns,
276+
directory-only patterns, and `**` globs. Negation can undo earlier Git rules,
277+
but cannot re-include an excluded parent directory or override built-in/config
278+
exclusions. Symlink entries are not followed or indexed.
278279

279-
**Dot files** are skipped by default (any path segment starting with `.`). whitelisted dot files (`.env`, `.prettierrc`, `.editorconfig`, etc.) are kept. Add more via `dot_whitelist` in config.
280+
**Dot files** are skipped by default (any path segment starting with `.`). whitelisted dot files (`.env.example`, `.prettierrc`, `.editorconfig`, etc.) remain eligible unless explicitly ignored. Add more via `dot_whitelist` in config.
280281

281282
## Data flow
282283

‎bench_engine.go‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ type EngineResult struct {
3030
HitAt1 bool `json:"hit_at_1"`
3131
HitAt3 bool `json:"hit_at_3"`
3232
HitAt5 bool `json:"hit_at_5"`
33-
NoiseRatio float64 `json:"noise_ratio"` // 1.0 - (relevantResults / totalResults)
33+
NoiseRatio float64 `json:"noise_ratio"` // 1.0 - (relevantResults / totalResults)
3434
}
3535

3636
// EngineSummary aggregates results for one engine across all cases.
@@ -99,6 +99,7 @@ func (unixFindEngine) Search(query string, expectAny []string, index *ContextInd
9999
if err != nil {
100100
return EngineResult{EngineName: "find", Error: fmt.Sprintf("build ignore map: %v", err)}
101101
}
102+
matcher := newIgnoreMatcher(ignored)
102103
cmd := exec.Command("find", index.RootPath, "-type", "f")
103104
cmd.Dir = index.RootPath
104105
out, runErr := cmd.Output()
@@ -117,7 +118,7 @@ func (unixFindEngine) Search(query string, expectAny []string, index *ContextInd
117118
if relErr != nil {
118119
continue
119120
}
120-
if shouldIgnorePath(rel, filepath.Base(rel), ignored) {
121+
if matcher.shouldIgnore(rel, filepath.Base(rel), false) {
121122
continue
122123
}
123124
if tokensMatchPath(rel, "", queryTokens) {
@@ -196,6 +197,7 @@ func (grepEngine) Search(query string, expectAny []string, index *ContextIndex,
196197
}
197198
queryTokens := tokenize(query)
198199
results := make([]string, 0)
200+
matcher := newIgnoreMatcher(ignored)
199201
err = filepath.WalkDir(index.RootPath, func(path string, d os.DirEntry, walkErr error) error {
200202
if walkErr != nil {
201203
return walkErr
@@ -207,7 +209,7 @@ func (grepEngine) Search(query string, expectAny []string, index *ContextIndex,
207209
if rel == "." {
208210
return nil
209211
}
210-
if shouldIgnorePath(rel, d.Name(), ignored) {
212+
if matcher.shouldIgnore(rel, d.Name(), d.IsDir()) {
211213
if d.IsDir() {
212214
return filepath.SkipDir
213215
}
@@ -388,8 +390,6 @@ func tokensMatchContent(content string, tokens []string) bool {
388390
return false
389391
}
390392

391-
392-
393393
func hasNullByte(data []byte, limit int) bool {
394394
if limit > len(data) {
395395
limit = len(data)
Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Search quality check — 2026-10-10
2+
3+
Compared the existing confidence-gap behavior with a narrower change: exact
4+
filename or exact symbol matches may stop at the first score cliff without
5+
padding to three results. Conceptual queries retain the existing three-result
6+
minimum when enough candidates exist. Candidates with no matching signal are
7+
excluded even when the caller sets `MinScore` to zero.
8+
9+
## Method
10+
11+
- Existing `docs/bench_cases.json`: 45 queries across six categories.
12+
- Same saved index for both runs, built from this repository with local symbol
13+
extraction and lexical synonyms. No LLM requests or paid services.
14+
- `ctxt` engine only, limit 10, minimum score 1, category reports enabled.
15+
- This checks the cutoff change; it does not measure LLM synonym quality or
16+
reproduce the separate monorepo used in `docs/contexting_feedback.md`.
17+
18+
## Results
19+
20+
| Metric | Before | After |
21+
| --- | ---: | ---: |
22+
| Hit@1 | 64.44% | 64.44% |
23+
| Hit@3 | 82.22% | 82.22% |
24+
| Hit@5 | 86.67% | 86.67% |
25+
| Recall | 86.67% | 86.67% |
26+
| Average results | 7.69 | 6.27 |
27+
| Average estimated tokens | 32.60 | 27.31 |
28+
| Average noise ratio | 80.69% | 67.85% |
29+
30+
Hit rates and recall were unchanged in every category. Exact filename queries
31+
already had 100% Hit@1. Vague intent remains the weakest category, with 14.29%
32+
Hit@1 on this offline index; these fixes do not resolve that vocabulary gap.
33+
34+
Applying the stricter cutoff to every query reduced recall to 84.44%, so that
35+
broader change was rejected.
36+
37+
## Validation
38+
39+
The existing `go test ./...` suite and `go vet ./...` passed. Temporary CLI
40+
fixtures also confirmed ordered Git negation, rooted/directory/recursive
41+
patterns, additive config exclusions, whitelisted dot files, symlink exclusion,
42+
newline filenames, hybrid type/minimum-score filtering, and watch bootstrap
43+
refreshing symbols after an edit with preserved modification time but new size.

‎docs/contexting_feedback.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Benchmark Feedback — ctxt search quality
22

3+
> Historical feedback from a separate monorepo. Exact basename boosts and symbol
4+
> scoring are implemented; confidence-gap behavior has since been refined. See
5+
> [the 2026-10-10 local evaluation](../bench_results/002_search_quality_2026-10-10.md)
6+
> for current results on this repository’s offline index.
7+
38
Ran `ctxt bench` with a 45-case file across a multi-language monorepo (Go + TypeScript + Svelte). 4 engines compared: ctxt, find, grep, combined. Overall results:
49

510
| Engine | Hit@1 | Hit@3 | Hit@5 | Avg Time | Avg Tokens | Noise |

‎gitignore.go‎

Lines changed: 4 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -91,20 +91,15 @@ func LoadGitignorePatterns(path string) ([]string, error) {
9191
patterns := make([]string, 0, 32)
9292
scanner := bufio.NewScanner(file)
9393
for scanner.Scan() {
94-
line := strings.TrimSpace(scanner.Text())
95-
if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, "!") {
96-
continue
97-
}
98-
line = strings.TrimPrefix(line, "./")
99-
line = strings.TrimPrefix(line, "/")
100-
line = strings.TrimSuffix(line, "/")
101-
if line == "" {
94+
line := strings.TrimSuffix(scanner.Text(), "\r")
95+
if line == "" || strings.HasPrefix(line, "#") {
10296
continue
10397
}
10498
patterns = append(patterns, line)
10599
}
106100
if err := scanner.Err(); err != nil {
107101
return nil, fmt.Errorf("read .gitignore: %w", err)
108102
}
109-
return dedupeStrings(patterns), nil
103+
// Order and duplicates matter: the last matching rule wins.
104+
return patterns, nil
110105
}

‎go.mod‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,24 @@ go 1.25.0
55
require (
66
github.com/BurntSushi/toml v1.6.0
77
github.com/fsnotify/fsnotify v1.9.0
8+
github.com/go-git/go-git/v5 v5.16.3
89
github.com/modelcontextprotocol/go-sdk v1.6.1
910
github.com/odvcencio/gotreesitter v0.23.1
1011
github.com/spf13/cobra v1.10.2
1112
golang.org/x/sys v0.41.0
1213
)
1314

1415
require (
16+
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
17+
github.com/go-git/go-billy/v5 v5.6.2 // indirect
1518
github.com/google/jsonschema-go v0.4.3 // indirect
1619
github.com/inconshreveable/mousetrap v1.1.0 // indirect
20+
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
1721
github.com/segmentio/asm v1.1.3 // indirect
1822
github.com/segmentio/encoding v0.5.4 // indirect
1923
github.com/spf13/pflag v1.0.9 // indirect
2024
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
25+
golang.org/x/net v0.50.0 // indirect
2126
golang.org/x/oauth2 v0.35.0 // indirect
27+
gopkg.in/warnings.v0 v0.1.2 // indirect
2228
)

‎go.sum‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,12 @@ github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2
33
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
44
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
55
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
6+
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
7+
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
8+
github.com/go-git/go-billy/v5 v5.6.2 h1:6Q86EsPXMa7c3YZ3aLAQsMA0VlWmy43r6FHqa/UNbRM=
9+
github.com/go-git/go-billy/v5 v5.6.2/go.mod h1:rcFC2rAsp/erv7CMz9GczHcuD0D32fWzH+MJAU+jaUU=
10+
github.com/go-git/go-git/v5 v5.16.3 h1:Z8BtvxZ09bYm/yYNgPKCzgWtaRqDTgIKRgIRHBfU6Z8=
11+
github.com/go-git/go-git/v5 v5.16.3/go.mod h1:4Ge4alE/5gPs30F2H1esi2gPd69R0C39lolkucHBOp8=
612
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
713
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
814
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
@@ -11,10 +17,20 @@ github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+
1117
github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
1218
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
1319
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
20+
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
21+
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
22+
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
23+
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
24+
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
25+
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
1426
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
1527
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
1628
github.com/odvcencio/gotreesitter v0.23.1 h1:EzvjNxzNr35TnlBK/3Bh0Hl6LRxHOl3JgRvVpGgDOI4=
1729
github.com/odvcencio/gotreesitter v0.23.1/go.mod h1:hBVkghd0paaYAVwd2087vfwdeU984bQbMo9LvpE0moo=
30+
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
31+
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
32+
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
33+
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
1834
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
1935
github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc=
2036
github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg=
@@ -27,10 +43,16 @@ github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An
2743
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
2844
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
2945
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
46+
golang.org/x/net v0.50.0 h1:ucWh9eiCGyDR3vtzso0WMQinm2Dnt8cFMuQa9K33J60=
47+
golang.org/x/net v0.50.0/go.mod h1:UgoSli3F/pBgdJBHCTc+tp3gmrU4XswgGRgtnwWTfyM=
3048
golang.org/x/oauth2 v0.35.0 h1:Mv2mzuHuZuY2+bkyWXIHMfhNdJAdwW3FuWeCPYN5GVQ=
3149
golang.org/x/oauth2 v0.35.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
3250
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
3351
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
3452
golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k=
3553
golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0=
3654
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
55+
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
56+
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
57+
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
58+
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=

0 commit comments

Comments
 (0)