detect_incremental() uses a single global manifest for all subdirectories — --update on one subfolder reports files from the last-graphed folder as deleted

[newwaysai]

Bug: detect_incremental() uses a single global manifest for all subdirectories — --update on one subfolder reports every file from the last-graphed folder as deleted

Version: graphifyy 0.3.28

What happened

Running the skill's --update flow on one subdirectory of a large repo (e.g. workflows/) after a previous full run had graphed a different subdirectory (e.g. experts/) causes deleted_files in the incremental-detect result to list hundreds of paths from that unrelated, previously-graphed folder.

Live case: --update on workflows/ (a workspace with ~15 top-level graphed subfolders, each run via /graphify <subfolder>) returned deleted_files containing 230+ paths under experts/rechts-kanzlei/... — none of which are under workflows/ and none of which were actually deleted. The last folder graphed before this run was experts/.

Root cause

In graphify/detect.py, the manifest path is a module-level constant:

_MANIFEST_PATH = "graphify-out/manifest.json"

load_manifest() / save_manifest() both default to this single flat path regardless of which root was passed to detect_incremental(root). Every /graphify <subfolder> --update run in a multi-subfolder workspace reads and writes the same manifest file, so the manifest only ever reflects whichever subfolder was graphed most recently. The comparison in detect_incremental():

current_files = {f for flist in full["files"].values() for f in flist}
deleted_files = [f for f in manifest if f not in current_files]

flags every manifest entry not under the current run's file set as deleted — including entries that were never under root to begin with.

Impact

If the caller (e.g. the skill's --update flow) prunes graph nodes whose source_file is in deleted_files, this can silently remove nodes belonging to an unrelated subfolder's graph, depending on which graph is currently loaded. In our case the prune step happened to target the just-loaded subfolder graph, which contained none of the flagged paths, so the visible symptom was only a wrong "N files deleted" count in the diff report — but the mechanism is a correctness bug regardless of the specific graph layout, and could delete real nodes in a different call pattern (e.g. running --update against the root graph, or against a subfolder whose graph overlaps the previously-graphed one).

Suggested fix

Scope the manifest path to root by default (e.g. graphify-out/<sanitized-root>/manifest.json, falling back to the existing flat path for the root-level . invocation to avoid a migration step for existing single-folder setups). Additionally, filter deleted_files to only include manifest entries that were ever under root, as defense in depth against a stale or manually merged manifest.

Happy to share a patch if useful — we worked around it locally by scoping _MANIFEST_PATH per-root and adding the under-root filter to deleted_files.

[harshaygadekar]

Confirmed and reproduced.

Root Cause

1. Flat default manifest: _MANIFEST_PATH in graphify/detect.py defaults to graphify-out/manifest.json. When graphing subfolder B (experts/) followed by an incremental scan on subfolder A (workflows/), subfolder B's manifest is loaded and re-anchored to subfolder A (workflows/expert.py), which does not exist and gets reported as deleted.
2. Double-nesting on workspace manifests: Running an incremental scan on a subfolder against a workspace manifest joins workspace-relative keys (workflows/workflow.py) with the subfolder root, resulting in invalid doubled paths (workflows/workflows/workflow.py).
3. Missing root boundary filter: detect_incremental() iterates through all loaded manifest rows without verifying whether each path actually falls within root.

Fix

1. Scope default manifest paths per scan root via manifest_path_for_root(root) (graphify-out/<slug>/manifest.json) when root is a subfolder, while preserving graphify-out/manifest.json for repo-level runs and falling back to the workspace manifest if no subfolder manifest exists yet.
2. In _to_absolute_from_storage(), walk parent directories if root / key does not exist on disk to resolve ancestor/workspace-relative keys without duplicating paths.
3. In detect_incremental(), skip entries outside root (if not _in_root(f): continue) so foreign paths are neither reported as deletions nor exclusions.

opened the PR for this issue, open to review!!

[safishamsi]

Resolved by #4118 (thanks @harshaygadekar), shipped in v0.9.77. Closing as fixed.