Is your feature request related to a problem? Please describe.
When using Joern as a persistent analysis server, re-indexing a project after a source file changes requires a full importCode rebuild. For large Java projects this takes several minutes (~10 min in our case), making interactive workflows impractical.
Describe the solution you'd like
An IncrementalCpgUpdater utility that re-indexes only the changed files in an existing CPG:
IncrementalCpgUpdater.updateFiles(cpg, project.inputPath, List("/path/to/Changed.java"))
Internally this would:
- Delete CPG nodes for the changed files (
File nodes + NamespaceBlock AST subtrees)
- Re-parse only those files via
AstCreationPass(sourcesOverride = Some(changedFiles))
- Re-run the standard post-passes (
OuterClassRefPass, TypeNodePass, TypeInferencePass, AstLinkerPass, ContainsEdgePass, StaticCallLinker, etc.)
We have a working prototype for javasrc2cpg with 8 unit tests passing (method add/remove, caller/callee edge add/remove). Performance: ~350ms per file vs ~10min for a full rebuild (~1700× speedup).
Describe alternatives you've considered
- Full
importCode rebuild on every change — too slow for interactive use
- Watching only specific node types — incomplete, misses call graph edges
Additional context
This is currently javasrc2cpg-specific. Happy to open a PR if the approach looks reasonable — would appreciate guidance on whether the HTTP endpoint (PUT /v1/cpg/:projectName/files) belongs in joern-cli or is out of scope.
Is your feature request related to a problem? Please describe.
When using Joern as a persistent analysis server, re-indexing a project after a source file changes requires a full
importCoderebuild. For large Java projects this takes several minutes (~10 min in our case), making interactive workflows impractical.Describe the solution you'd like
An
IncrementalCpgUpdaterutility that re-indexes only the changed files in an existing CPG:Internally this would:
Filenodes +NamespaceBlockAST subtrees)AstCreationPass(sourcesOverride = Some(changedFiles))OuterClassRefPass,TypeNodePass,TypeInferencePass,AstLinkerPass,ContainsEdgePass,StaticCallLinker, etc.)We have a working prototype for
javasrc2cpgwith 8 unit tests passing (method add/remove, caller/callee edge add/remove). Performance: ~350ms per file vs ~10min for a full rebuild (~1700× speedup).Describe alternatives you've considered
importCoderebuild on every change — too slow for interactive useAdditional context
This is currently
javasrc2cpg-specific. Happy to open a PR if the approach looks reasonable — would appreciate guidance on whether the HTTP endpoint (PUT /v1/cpg/:projectName/files) belongs injoern-clior is out of scope.