Pre-flight checks
What happened?
An extractor can mint a node for a file that is not in the checkout: the target of an unresolved dynamic import (import('./queue.js')), or a project a .sln / .slnx / <ProjectReference> names. The node carries the missing path as its source_file, but it is the referring file's output (label ./queue.js, edges only from the referrer).
On a changed-files rebuild (_rebuild_code(changed_paths=...), used by the post-commit hook and graphify watch), the corpus sweep in _reconcile_existing_graph reads that missing path as a deleted source and evicts the node. The unchanged referrer's preserved edges to it then dangle and are dropped. A full rebuild of the same tree keeps both, so after every hook or watch rebuild the graph silently differs from a full rebuild, and it flips back on the next graphify update.
This goes against CONTRIBUTING ("Cross-file edges: must be preserved during incremental updates") and against the intent of #1795 ("require deletion evidence before evicting a missing source"): a path that was never scanned has no deletion evidence.
On graphify's own repo (975 files, v8 5c7b847), editing one unrelated file and running the changed-files rebuild loses 8 nodes and 15 edges compared with a full rebuild of the same tree: the placeholders for tests/fixtures/dynamic_import.ts's missing ./queue.js, ./mayaEngine.js, ./staticHelper, and the .csproj projects sample.sln, sample.slnx and sample.csproj reference.
Related to #3570 (same symptom class); the commenters there found that cause fixed by 32b209a, and this is a different mechanism that still reproduces on 0.9.77.
Steps to reproduce
cat > repro.py <<'EOF'
import json, tempfile
from pathlib import Path
from graphify.watch import _rebuild_code
def placeholder(c):
g = json.loads((c / "graphify-out/graph.json").read_text(encoding="utf-8"))
nodes = [n["id"] for n in g["nodes"] if n.get("source_file", "").endswith("queue.js")]
edges = [(e["source"], e["relation"]) for e in g["links"] if e["target"] in nodes]
return nodes, edges
if __name__ == "__main__":
c = Path(tempfile.mkdtemp()) / "corpus"; c.mkdir()
(c / "app.ts").write_text("export async function load() {\n return import('./queue.js');\n}\n")
(c / "other.ts").write_text("export function other() {\n return 1;\n}\n")
_rebuild_code(c, no_cluster=True, acquire_lock=False, force=True)
print("full rebuild: ", placeholder(c))
(c / "other.ts").write_text("export function other() {\n return 2;\n}\n")
_rebuild_code(c, changed_paths=[Path("other.ts")], no_cluster=True, acquire_lock=False, force=True)
print("changed-files (other.ts): ", placeholder(c))
_rebuild_code(c, no_cluster=True, acquire_lock=False, force=True)
print("full rebuild, same tree: ", placeholder(c))
EOF
PYTHONHASHSEED=0 python repro.pyError output or graph output
full rebuild: (['queue'], [('app_load', 'imports_from'), ('app', 'dynamic_import')])
changed-files (other.ts): ([], [])
full rebuild, same tree: (['queue'], [('app_load', 'imports_from'), ('app', 'dynamic_import')])
On graphify's own repo, after a one-file edit: changed-files rebuild 18,617 nodes / 37,920 edges vs full rebuild of the same tree 18,625 / 37,935.
Graphify version
0.9.77 (5c7b847)
Operating System
Windows
Python Version
3.12
Installation Method
built from source (git clone)
Additional Environment Details
AST-only, no provider environment variables, PYTHONHASHSEED=0. A clean checkout of v8 at 5c7b847 reproduces it. Found while comparing changed-files and full rebuilds of graphify's own repo.
Additional context
Fix ready: use the previous scan (manifest.json, rewritten only after reconcile) as the deletion evidence. A missing path that was in the last scan is evicted as before; one that was never scanned is kept while a referrer that is not re-extracted this run still points at it. No manifest: old behaviour. PR right after this.
Pre-flight checks
What happened?
An extractor can mint a node for a file that is not in the checkout: the target of an unresolved dynamic import (
import('./queue.js')), or a project a.sln/.slnx/<ProjectReference>names. The node carries the missing path as itssource_file, but it is the referring file's output (label./queue.js, edges only from the referrer).On a changed-files rebuild (
_rebuild_code(changed_paths=...), used by the post-commit hook andgraphify watch), the corpus sweep in_reconcile_existing_graphreads that missing path as a deleted source and evicts the node. The unchanged referrer's preserved edges to it then dangle and are dropped. A full rebuild of the same tree keeps both, so after every hook or watch rebuild the graph silently differs from a full rebuild, and it flips back on the nextgraphify update.This goes against CONTRIBUTING ("Cross-file edges: must be preserved during incremental updates") and against the intent of #1795 ("require deletion evidence before evicting a missing source"): a path that was never scanned has no deletion evidence.
On graphify's own repo (975 files, v8 5c7b847), editing one unrelated file and running the changed-files rebuild loses 8 nodes and 15 edges compared with a full rebuild of the same tree: the placeholders for
tests/fixtures/dynamic_import.ts's missing./queue.js,./mayaEngine.js,./staticHelper, and the.csprojprojectssample.sln,sample.slnxandsample.csprojreference.Related to #3570 (same symptom class); the commenters there found that cause fixed by 32b209a, and this is a different mechanism that still reproduces on 0.9.77.
Steps to reproduce
Error output or graph output
Graphify version
0.9.77 (5c7b847)
Operating System
Windows
Python Version
3.12
Installation Method
built from source (git clone)
Additional Environment Details
AST-only, no provider environment variables, PYTHONHASHSEED=0. A clean checkout of
v8at5c7b847reproduces it. Found while comparing changed-files and full rebuilds of graphify's own repo.Additional context
Fix ready: use the previous scan (manifest.json, rewritten only after reconcile) as the deletion evidence. A missing path that was in the last scan is evicted as before; one that was never scanned is kept while a referrer that is not re-extracted this run still points at it. No manifest: old behaviour. PR right after this.