Repository navigation
feat(config): let projects declare languages for ambiguous extensions via .graphifyrc - #3789
andrebrait wants to merge 8 commits into
Conversation
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (ef7a41c) onto upstream/v8.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Graphify review — findings
Adds a .graphifyrc project config that lets a repo declare what an ambiguous extension means to it (e.g. language.inc=php), driving file classification, extractor and grammar/resolution selection, and import resolution for the declared suffixes while keeping native suffixes in precedence. Salts the AST/semantic cache by the effective language via _salted_key so a file remapped to another language never reuses an entry built under the old extractor, and records language in the manifest so adding, changing, or removing a declaration invalidates affected incremental entries even when the source bytes are unchanged. Activates the overrides per scan root in detect, save_manifest, and extraction, and routes C# file detection through _is_cs_file so declared C# suffixes are recognized.
Worth a look
- Process-global language overrides race between concurrent extract calls —
graphify/extract.py:7285· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- detect_incremental uses language override salt without activating the requested root —
graphify/detect.py:2543· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Language overrides can leak into standalone classification —
graphify/detect.py:526· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Objective-C aliases resolve to Objective-C++ suffix —
graphify/rcfile.py:49· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- extract() now writes language override status to stdout —
graphify/extract.py:7285· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3984 functions depend on the 1440 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 710 callers, 50 callees - new:
_rebuild_code()— 144 callers, 55 callees - new:
detect()— 114 callers, 16 callees - new:
save_semantic_cache()— 63 callers, 9 callees - new:
save_manifest()— 42 callers, 13 callees - new:
_extract_generic()— 18 callers, 29 callees - new:
extract_js()— 87 callers, 5 callees - new:
load_cached()— 53 callers, 8 callees - …and 106 more — each is listed as a finding
Verification — 3984 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 3746 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
306 of 306 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/test_affected_cli.py— impact, full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_agents_platform.py— impact, full-run-safetytests/test_analyze.py— full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— impact, full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— impact, full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— impact, changed-test, full-run-safetytests/test_benchmark.py— full-run-safetytests/test_benchmark_raw_graph.py— full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_merge_dedup_scope.py— full-run-safetytests/test_build_merge_hyperedges_and_prune.py— full-run-safetytests/test_build_merge_shrink_guard.py— full-run-safetytests/test_builtin_global_type_refs.py— impact, full-run-safetytests/test_cache.py— impact, full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— impact, full-run-safetytests/test_cargo_missing_manifest.py— full-run-safetytests/test_carried_hyperedge_remap.py— full-run-safetytests/test_case_sensitive_resolution.py— impact, full-run-safetytests/test_charmap_encoding.py— impact, full-run-safetytests/test_chunking.py— impact, full-run-safetytests/test_cjs_module_extension.py— impact, full-run-safetytests/test_claude_cli_backend.py— impact, full-run-safetytests/test_claude_md.py— full-run-safetytests/test_cli_broken_pipe.py— full-run-safetytests/test_cli_export.py— full-run-safetytests/test_cli_help.py— full-run-safetytests/test_cluster.py— full-run-safetytests/test_cobol_extractor.py— impact, full-run-safetytests/test_codebuddy.py— impact, full-run-safetytests/test_community_hub_labels.py— full-run-safetytests/test_community_labels_skill.py— full-run-safetytests/test_confidence.py— full-run-safetytests/test_corrupt_graph_json.py— full-run-safetytests/test_cpp_nested_and_cli.py— impact, full-run-safetytests/test_cpp_objc_cross_file_calls.py— impact, full-run-safetytests/test_cpp_preprocess.py— full-run-safetytests/test_cross_extension_reexport_self_cycle.py— impact, full-run-safetytests/test_cross_language_call_resolution.py— impact, full-run-safetytests/test_cross_repo_external_call_guards.py— impact, full-run-safetytests/test_cross_repo_member_calls.py— impact, full-run-safety- … and 256 more
non-code file(s) changed (
README.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
README.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
· 11 grounded finding(s) anchored inline below; 103 more finding(s) on lines outside this diff (see the check run).
| return hashlib.sha256(f"{h}:{salt}".encode("utf-8")).hexdigest() | ||
|
|
||
|
|
||
| def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", |
There was a problem hiding this comment.
load_cached()
fans out to 8 callees (efferent coupling); 53 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -1070,9 +1083,13 @@ def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", | |||
|
|
|||
| def save_cached(path: Path, result: dict, root: Path = Path("."), kind: str = "ast", | |||
There was a problem hiding this comment.
save_cached()
fans out to 8 callees (efferent coupling); 27 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -1737,6 +1740,9 @@ def _resolves_under_root(path: Path, root: Path) -> bool: | |||
|
|
|||
| def detect(root: Path, *, follow_symlinks: bool | None = None, google_workspace: bool | None = None, extra_excludes: list[str] | None = None, cache_root: Path | None = None, gitignore: bool = True) -> dict: | |||
There was a problem hiding this comment.
detect()
fans out to 16 callees (efferent coupling); 114 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -486,18 +500,18 @@ def _repoint_python_package_imports(paths, all_nodes, all_edges, root) -> None: | |||
|
|
|||
| def _repoint_python_sibling_imports(paths, all_nodes, all_edges, root) -> None: | |||
There was a problem hiding this comment.
_repoint_python_sibling_imports()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -2003,7 +2017,7 @@ def extract_python(path: Path, *, root: Path | None = None) -> dict: | |||
|
|
|||
| def extract_js(path: Path) -> dict: | |||
There was a problem hiding this comment.
extract_js()
87 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| from graphify.rcfile import effective_suffix | ||
|
|
||
|
|
||
| def extract_ocaml(path: Path) -> dict: |
There was a problem hiding this comment.
extract_ocaml()
fans out to 8 callees (efferent coupling); 13 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return tuple(sorted(suffixes)) | ||
|
|
||
|
|
||
| def _resolve_js_import_path(candidate: Path) -> Path: |
There was a problem hiding this comment.
_resolve_js_import_path()
high coupling complexity (Ca·Ce = 15).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -1970,7 +1991,7 @@ def _ts_walk_class_members(class_node, source: bytes, path: Path, class_nid: str | |||
| def _collect_js_symbol_resolution_facts(paths: list[Path], facts: _SymbolResolutionFacts) -> None: | |||
There was a problem hiding this comment.
_collect_js_symbol_resolution_facts()
fans out to 27 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ) from None | ||
|
|
||
|
|
||
| def load_graphifyrc(root: Path) -> dict: |
There was a problem hiding this comment.
load_graphifyrc()
10 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return dict(_active()) | ||
|
|
||
|
|
||
| def activate_language_overrides(root: Path) -> dict[str, str]: |
There was a problem hiding this comment.
activate_language_overrides()
26 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (ef7a41c) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (ef7a41c) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (de53a62) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (de53a62) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (079b121) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (79a451e) onto upstream/v8.
ef7a41c to
79a451e
Compare
79a451e to
f9136b7
Compare
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (f9136b7) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (f9136b7) onto upstream/v8.
f9136b7 to
ab2c70b
Compare
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (ab2c70b) onto upstream/v8.
ab2c70b to
e5b5e55
Compare
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (e5b5e55) onto upstream/v8.
e5b5e55 to
f50397a
Compare
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (f50397a) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (f50397a) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (f50397a) onto upstream/v8.
…Graphify-Labs#2961) `.inc` is hard-mapped to the Pascal extractor, but it is "include file" in whatever language a project happens to use: PHP on pfSense, Pascal in a Delphi tree, SQL or assembly elsewhere. A PHP `.inc` parsed as Pascal does not fail — it yields a handful of incidental nodes, so the graph looks populated while the shipped runtime is missing from it (7 nodes instead of 471 on the reporter's file). Hardcoding one more extension (Graphify-Labs#1042's fix) cannot generalise, because `.inc` has no single correct global meaning. `.graphifyrc` gains `language.<ext>=<language | .ext>`: language.inc=php language.tpl=.ts The declaration reaches every place graphify keys a decision on the suffix: * detect.classify_file — a declared extension counts as code * extract._get_extractor — dispatch, ahead of the .h/.m sniffs * _lang_is_case_insensitive / — cross-file resolution rules _lang_family / the PHP pass * resolver_registry — the declared language's resolvers wake * cache.load_cached/save_cached — a `salt` folds the target language into the key, so the same bytes cached under the old extractor are never served for the new one * the extraction pool — an initializer forwards the mapping to workers, which start with empty module state under `spawn` The parser moves from hooks.py into a dependency-free graphify.rcfile so detect/extract (and the workers) can read it; hooks._load_graphifyrc keeps its name and behaviour. detect() and extract() activate `<root>/.graphifyrc` themselves, so the CLI, `update`/`watch`, the hooks, the MCP server, and the skill runbook all pick it up with no further wiring. A malformed file is reported once on stderr and the scan continues with the defaults. (cherry picked from commit 3740fbf)
…ocess-global (Graphify-Labs#2961) Review follow-up: the active overrides lived in a module-global dict, so two extractions running concurrently in one process (an MCP server, a threaded harness) with different roots would clobber each other — one scan's activate_language_overrides overwrote the other's while effective_suffix / classify_file / cache_salt read it, cross-contaminating which extractor an ambiguous .inc/.m/.h file got. The overrides are now thread-local. The dominant paths are unchanged: the CLI runs on the main thread, and a spawn pool worker sets its own copy in _worker_init on the same main thread that runs _extract_single_file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JJbLfztSxm2tH5cJwBbe9q (cherry picked from commit 1ad08a9)
(cherry picked from commit 9be01cc)
(cherry picked from commit 5c8aebc)
(cherry picked from commit 4815743)
(cherry picked from commit a59563e)
Upstream 0.9.6x added a real (optional-dependency-gated) .r extractor since this test was first written, so `language.tpl=r` now hits the "dependency is missing" warning instead of "no AST extractor". Swap to .ets (ArkTS), a recognized code extension with no registered extractor today, to keep exercising the declared-but-unsupported path.
f50397a to
c041c90
Compare
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (c041c90) onto upstream/v8.
…or ambiguous extensions via .graphifyrc Squashed from origin/feat/rcfile-language-overrides (c041c90) onto upstream/v8.
This branch has not been deployed
2 participants
Carries the original
.graphifyrclanguage-override work from #3075 by @abhay-codes07 onto the currentv8base, preserving its commit authorship and messages, then layers the reviewed follow-up fixes developed while maintaining it in the pfBlockerNG fork.Relationship to #3075
v8conflict/fixture adaptation into one tested branch. It is a current-base integration candidate, not a claim of original authorship; maintainers can choose whichever review/merge shape is easiest.The automated integration rebuild consumes open PRs against this repository. Keeping the full stack represented here prevents the downstream build from dropping the follow-up fixes while #3075 remains open.
The problem
.incis hard-mapped to the Pascal extractor, but it means "include file" in whatever language a project actually uses: PHP on pfSense/pfBlockerNG, Pascal in a Delphi tree, SQL or assembly elsewhere. A PHP.incparsed as Pascal doesn't fail loudly — it yields a handful of incidental nodes, so the graph looks populated while the real code is missing from it. The same ambiguity affects.h(C vs C++) and.m(Objective-C vs MATLAB).pfBlockerNG depends on this: its
.graphifyrcsetslanguage.inc=phpto parse 21 PHP include files (pfblockerng.inc,pfblockerng_extra.inc, etc.) into the tracked knowledge graph. Without the override, those 21 files contribute ~29 incidental (mis-parsed-as-Pascal) nodes; with it, they contribute the real corpus.The change
graphify/rcfile.py(new): owns all.graphifyrcparsing —viz_node_limit(unchanged contract) pluslanguage.<ext>=<language-or-explicit-extension>. Unknown keys are ignored (forward-compatible); malformed lines/values raiseValueErrornaming the offending line..inc/.m/.hgets.graphify/hooks.py's_load_graphifyrcnow delegates torcfile.load_graphifyrc, keepingviz_node_limitworking for existing callers/tests unchanged.detect.py,extract.py,symbol_resolution.py,extractors/{resolution,csharp,ocaml,robot}.py,resolver_registry.py, andwatch.pyall consulteffective_suffix()so the override is honored consistently in classification, extraction, cross-file import/call resolution, and live graph refresh (edit.graphifyrc→ the nextgraphify watch/updatere-dispatches affected files).language.tpl=.ets) are validated against the known code/doc extension set at parse time.Testing
tests/test_language_overrides.py,tests/test_language_override_lifecycle.py,tests/test_pr10_csharp_overrides.py,tests/test_pr10_resolution_overrides.py,tests/test_pr10_watch_overrides.py(new, cover the override contract end to end: parsing, thread-local activation, extraction/import/resolution dispatch, and watch-triggered re-extraction on.graphifyrcchanges).tests/test_backend_extras.py,tests/test_extract.py,tests/test_hooks.py(existing upstream coverage, kept green —test_hooks.pyin particular is upstream's own.graphifyrc/viz_node_limitcontract test).Rebased cleanly onto
v8(0.9.67) with one manual conflict resolution ingraphify/extractors/resolution.py::_collect_python_symbol_resolution_facts: upstream's absolute-import work addedambiguous_python_modules = ambiguous_python_modules or set()immediately before thepy_pathsfilter that this stack changes frompath.suffix == ".py"toeffective_suffix(path) == ".py". Both changes are independent and were combined (kept upstream's normalization, applied the override-aware suffix check). One pre-existing fixture intest_language_overrides.pywas updated: it declared.tpl=rto exercise the "declared-but-no-extractor" warning path, but upstream 0.9.6x added a real (optional-dependency-gated).rextractor since this stack was first written, so that scenario now hits the "dependency is missing" warning instead of "no AST extractor". Swapped the fixture to.tpl=.ets(ArkTS, a recognized code extension with no registered extractor today), which preserves the original invariant under test.Functional verification
Rebuilding pfBlockerNG's tree with this branch (
graphify update .) against its.graphifyrc(language.inc=php) produces 940 nodes across all 21.incfiles (9 of them contributing more than one node), versus ~29 nodes without the override.