Skip to content

Commit f792572

Browse files
Technologicatclaude
andcommitted
analyzer: module-level name bindings produce graph Nodes
Generalize the analyzer's notion of "defined Node" so that every named entity reachable from outside its definition site has a Node — not just classes, functions, modules, and imports. Until now, module-level plain assignments (`CONSTANT = 42`, `LOGGER = ...`, `store = _NS()`) were tracked only as scope-local set_value bindings, with no Node in the graph. Cross-module imports contracted to wildcards, and the #127 attribute-fallback had to climb all the way to the enclosing module since the binding itself wasn't an addressable Node. Now `_bind_target` creates a defined Flavor.NAME Node at the LHS dotted path whenever the current scope is a module or class. Function-locals stay as scope-only set_value bindings (promoting every loop variable to a Node would clutter the graph). Consequences: - `from mymod import CONSTANT` resolves to the actual NAME Node rather than contracting to a wildcard. - The #127 fallback for `store.dataset` lands on `mymod.store` (the binding) rather than the enclosing module — strictly more specific. For the simple-attribute case the "fallback" no longer climbs at all, since obj_node itself is already defined; the same code path still fires but is now redundant with what the obj_node already provides. The chained-access case still genuinely climbs through undefined intermediate ATTRIBUTE Nodes. - Edgeless NAME Nodes (module constants that nobody imports) are suppressed from the rendered output by default in visgraph, so the default visual density is unchanged. The Node still exists in the analyzer's graph for cross-module resolution. Tests: - The three #127 regression tests now assert the more-specific edge target (`namespace_module.store` instead of `namespace_module`). The simple-case tests are renamed to `_emits_edge_to_binding`, dropping the "falls_back" framing since obj_node is now already defined; the chained-case test keeps "climbs_to_defined_ancestor" because the climb through undefined intermediates is still genuine. - Four new tests in test_features.py cover the prequisite directly: module-level binding creates a defined NAME Node, function-local does not, cross-module constant import resolves to the NAME Node, visgraph suppresses edgeless NAME Nodes. Prequisite to #129. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 17ebc0f commit f792572

6 files changed

Lines changed: 174 additions & 29 deletions

File tree

CHANGELOG.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,20 @@
11
# Changelog
22

3-
## 2.5.1 (in progress)
3+
## 2.6.0 (in progress)
4+
5+
### New features
6+
7+
- **Module-level name bindings now produce graph Nodes.** Every module-level assignment (e.g. `CONSTANT = 42`, `LOGGER = logging.getLogger(__name__)`, `store = _NS()`) creates a defined `Flavor.NAME` Node at the bound dotted path. `from mymod import x` now resolves to the actual binding instead of contracting to a wildcard, and the #127 attribute-fallback lands on the specific binding rather than climbing all the way to the enclosing module. Function-locals are unchanged — they stay as scope-only bindings to keep the graph readable. Edgeless NAME Nodes (module constants nobody imports) are suppressed from the rendered output by default; they remain in the analyzer's graph for cross-module resolution.
48

59
### Bug fixes
610

711
- **Cross-module attribute reads on namespace-style modules now produce uses edges.** A module whose public surface is a runtime-built object (`SimpleNamespace`, `unpythonic.env.env`, a small `class _NS: pass; store = _NS()` shim) used to appear as an isolated node even when it was central to the subsystem — every `store.dataset` access landed on a synthetic ATTRIBUTE node that visgraph dropped as undefined. The analyzer now also emits an edge to the immediate defined parent of the obj (typically the exporting module) when the attribute itself can't be resolved. Symmetric for attribute writes (`store.flag = value`). One-level only — does not climb through unanalyzed packages, and within-scope self-references are suppressed (a method reading an undefined attribute on its own class, or a function reading module-level state in its own module, is just normal scoping). Generalizes the existing class-fallback (Enum members, class constants) introduced in 2.4.0. (#127)
812
- **Advisory when `infer_root` may have misidentified the package root.** Two ambiguous situations now emit a warning suggesting `--root`: (1) inference walked up at least one package level and stopped at a directory that has neither `__init__.py` nor a project-root marker (`pyproject.toml`, `setup.py`, `setup.cfg`) — consistent with a top-level PEP 420 namespace package; (2) inference didn't walk up at all but the input directory's parent has `__init__.py` — consistent with the user feeding pyan the contents of a namespace subpackage (e.g. `pyan3 pkg/sub_ns/*.py` where `sub_ns/` has no `__init__.py`), which would otherwise silently produce bare module names and broken relative imports. Auto-walking further is unsafe — the same filesystem shapes also occur for workspace directories like `tests/` or `examples/` — so the choice is left to the user. The `--root` help text now also mentions the namespace-package case explicitly. (#128)
913

14+
### Internal
15+
16+
- **Flavor rename:** `Flavor.NAMESPACE` (synthetic structural marker for module/class/function scope bookkeeping) is now `Flavor.SCOPE`. The previous name is being freed up for an upcoming `Flavor.NAMESPACE_OBJECT` representing a runtime namespace value (env, SimpleNamespace, …). The Node represents the scope; the `Scope` class implements one — same concept at two layers.
17+
1018

1119
---
1220

pyan/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/usr/bin/env python3
22

3-
__version__ = "2.5.1-dev"
3+
__version__ = "2.6.0-dev"
44

55
from .main import create_callgraph, main # noqa: F401
66
from .modvis import create_modulegraph # noqa: F401

pyan/analyzer.py

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1763,6 +1763,7 @@ def _bind_target(self, target, value):
17631763
"""
17641764
if isinstance(target, ast.Name):
17651765
self.set_value(target.id, value)
1766+
self._maybe_define_name_node(target)
17661767
elif isinstance(target, ast.Attribute):
17671768
try:
17681769
if self.set_attribute(target, value):
@@ -1792,6 +1793,39 @@ def _bind_target(self, target, value):
17921793
elif isinstance(target, ast.arg):
17931794
self.set_value(target.arg, value)
17941795

1796+
def _maybe_define_name_node(self, name_target):
1797+
"""Create a defined ``Flavor.NAME`` Node for an ``ast.Name`` binding
1798+
target if currently in a module or class scope.
1799+
1800+
The graph's notion of "defined Node" tracks named entities reachable
1801+
from outside their definition site. Module-level and class-level
1802+
bindings are reachable (via ``from mymod import x`` or ``Class.x``)
1803+
and so deserve a Node so that cross-module imports and attribute
1804+
accesses can resolve to the actual binding rather than degrading to
1805+
a wildcard or to #127's coarser module-level fallback.
1806+
1807+
Function-locals (and synthetic anonymous scopes — comprehensions,
1808+
lambdas — which symtable reports as ``"function"``) are not
1809+
externally addressable; promoting every loop variable to a Node
1810+
would only clutter the graph. They stay scope-only, set via
1811+
``set_value``.
1812+
1813+
Method/function/class definitions are flavored separately by their
1814+
own visitors (``visit_FunctionDef``, ``visit_ClassDef``) and don't
1815+
come through this path.
1816+
"""
1817+
if not self.scope_stack:
1818+
return
1819+
scope = self.scope_stack[-1]
1820+
if scope.type not in ("module", "class"):
1821+
return
1822+
from_node = self.get_node_of_current_namespace()
1823+
ns = from_node.get_name()
1824+
to_node = self.get_node(ns, name_target.id, name_target, flavor=Flavor.NAME)
1825+
if self.add_defines_edge(from_node, to_node):
1826+
self.logger.info(f"Def from {from_node} to NAME {to_node}")
1827+
self.associate_node(to_node, name_target, self.filename)
1828+
17951829
@staticmethod
17961830
def _collect_target_names(target, names):
17971831
"""Collect all Name identifiers from an assignment target AST node."""

pyan/visgraph.py

Lines changed: 34 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -173,11 +173,31 @@ def make_tooltip(n):
173173
logger = logger or logging.getLogger(__name__)
174174

175175
# collect and sort defined nodes
176+
#
177+
# NAME-flavored Nodes with no incoming or outgoing uses edges are
178+
# suppressed by default. Module-level bindings (e.g. ``__version__``,
179+
# ``LOGGER``) become defined NAME Nodes during analysis to make them
180+
# addressable for cross-module imports, but if nothing actually
181+
# imports or uses them, they only add visual noise. Their defines
182+
# edge from the enclosing module is *always* present by construction
183+
# and so cannot indicate use; we look at uses_edges only.
184+
from .node import Flavor # noqa: PLC0415 -- avoid circular import at module level
185+
named_with_uses = set()
186+
for from_node, to_nodes in visitor.uses_edges.items():
187+
if from_node.flavor == Flavor.NAME:
188+
named_with_uses.add(from_node)
189+
for to_node in to_nodes:
190+
if to_node.flavor == Flavor.NAME:
191+
named_with_uses.add(to_node)
192+
176193
visited_nodes = []
177194
for name in visitor.nodes:
178195
for node in visitor.nodes[name]:
179-
if node.defined:
180-
visited_nodes.append(node)
196+
if not node.defined:
197+
continue
198+
if node.flavor == Flavor.NAME and node not in named_with_uses:
199+
continue
200+
visited_nodes.append(node)
181201
visited_nodes.sort(key=lambda x: (x.namespace, x.name))
182202

183203
def find_filenames():
@@ -254,17 +274,21 @@ def find_filenames():
254274
#
255275
color = "#838b8b" if draw_defines else "#ffffff00"
256276
for n in visitor.defines_edges:
257-
if n.defined:
258-
for n2 in visitor.defines_edges[n]:
259-
if n2.defined:
260-
root_graph.edges.append(VisualEdge(nodes_dict[n], nodes_dict[n2], "defines", color))
277+
if n not in nodes_dict:
278+
continue
279+
for n2 in visitor.defines_edges[n]:
280+
if n2 not in nodes_dict:
281+
continue
282+
root_graph.edges.append(VisualEdge(nodes_dict[n], nodes_dict[n2], "defines", color))
261283

262284
if draw_uses:
263285
color = "#000000"
264286
for n in visitor.uses_edges:
265-
if n.defined:
266-
for n2 in visitor.uses_edges[n]:
267-
if n2.defined:
268-
root_graph.edges.append(VisualEdge(nodes_dict[n], nodes_dict[n2], "uses", color))
287+
if n not in nodes_dict:
288+
continue
289+
for n2 in visitor.uses_edges[n]:
290+
if n2 not in nodes_dict:
291+
continue
292+
root_graph.edges.append(VisualEdge(nodes_dict[n], nodes_dict[n2], "uses", color))
269293

270294
return root_graph

tests/test_features.py

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -820,3 +820,79 @@ def test_depth_no_self_edges():
820820

821821
for n, edges in v.uses_edges.items():
822822
assert n not in edges, f"Self-edge on {n.get_name()}"
823+
824+
825+
# --- Module-level NAME Node-ification ---
826+
#
827+
# Every named entity reachable from outside its definition site is a Node.
828+
# Module-level and class-level bindings are reachable; function-locals are not.
829+
830+
def _name_nodes_at(visitor, namespace):
831+
"""Return defined NAME-flavored Nodes at *namespace*."""
832+
from pyan.node import Flavor
833+
return [
834+
n for ns in visitor.nodes.values() for n in ns
835+
if n.defined and n.flavor == Flavor.NAME and n.namespace == namespace
836+
]
837+
838+
839+
def test_module_level_assignment_creates_defined_name_node():
840+
"""``mymod.x = ...`` at module level produces a defined ``Flavor.NAME``
841+
Node at ``mymod.x``, with a defines edge from the module Node."""
842+
v = CallGraphVisitor.from_sources([
843+
("CONSTANT = 42\nLOGGER = object()\n", "mymod"),
844+
])
845+
name_nodes = {n.name for n in _name_nodes_at(v, "mymod")}
846+
assert "CONSTANT" in name_nodes
847+
assert "LOGGER" in name_nodes
848+
849+
850+
def test_function_local_does_not_create_name_node():
851+
"""Function-local bindings stay as scope-only ``set_value`` and do not
852+
produce graph Nodes — they aren't externally addressable and would
853+
only clutter the graph."""
854+
v = CallGraphVisitor.from_sources([
855+
("def f():\n local = 1\n return local\n", "mymod"),
856+
])
857+
name_nodes = {n.name for n in _name_nodes_at(v, "mymod.f")}
858+
assert "local" not in name_nodes
859+
860+
861+
def test_cross_module_constant_import_resolves_to_name_node():
862+
"""``from mymod import CONSTANT`` should bind to the actual NAME Node
863+
at ``mymod.CONSTANT``, not contract to a wildcard. This is the
864+
precision win from making module-level bindings addressable Nodes."""
865+
v = CallGraphVisitor.from_sources([
866+
("CONSTANT = 42\n", "constants"),
867+
("from constants import CONSTANT\n\ndef use():\n return CONSTANT\n", "consumer"),
868+
])
869+
use_uses = {n.get_name() for n in v.uses_edges.get(
870+
v.get_node("consumer", "use"), set()
871+
)}
872+
assert "constants.CONSTANT" in use_uses
873+
874+
875+
def test_visgraph_suppresses_edgeless_name_nodes():
876+
"""NAME Nodes with no incoming or outgoing uses edges should not appear
877+
in the visgraph output (visual-density default). The Node still exists
878+
in the analyzer's graph for cross-module resolution; only the rendered
879+
output filters it."""
880+
from pyan.visgraph import VisualGraph
881+
v = CallGraphVisitor.from_sources([
882+
# UNUSED has no uses edges anywhere; USED is imported by consumer.
883+
("UNUSED = 1\nUSED = 2\n", "constants"),
884+
("from constants import USED\n\ndef f():\n return USED\n", "consumer"),
885+
])
886+
vg = VisualGraph.from_visitor(v, options={"draw_defines": True, "draw_uses": True})
887+
888+
def collect(graph, out):
889+
for n in graph.nodes:
890+
out.add(n.label)
891+
for sg in graph.subgraphs:
892+
collect(sg, out)
893+
labels = set()
894+
collect(vg, labels)
895+
assert any("USED" in lab for lab in labels), f"USED should appear; saw {labels}"
896+
assert not any("UNUSED" in lab for lab in labels), (
897+
f"UNUSED has no uses edges and should be suppressed; saw {labels}"
898+
)

tests/test_regressions.py

Lines changed: 20 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -407,36 +407,39 @@ def _issue127_visitor(*basenames):
407407
return CallGraphVisitor(filenames, logger=logging.getLogger())
408408

409409

410-
def test_issue127_attr_read_falls_back_to_module():
410+
def test_issue127_unresolved_attr_read_emits_edge_to_binding():
411411
"""Reading ``store.dataset`` where ``dataset`` isn't statically known
412-
should produce a uses edge to the nearest defined ancestor of the
413-
unresolved chain — here, the ``namespace_module`` module itself
414-
(since ``store`` is an IMPORTEDITEM with ``defined=False``). Without
415-
this fallback, namespace-style modules become invisible in the graph."""
412+
should still produce a uses edge to ``namespace_module.store`` (the
413+
binding), so the cross-module coupling stays visible. The binding is
414+
a defined ``Flavor.NAME`` Node, so the edge lands on it directly —
415+
no climb needed. (Pre-#129-prequisite, the binding wasn't a Node and
416+
the edge had to fall back to the enclosing module.)"""
416417
v = _issue127_visitor("namespace_module.py", "consumer.py")
417418
uses = get_in_dict(v.uses_edges, "consumer.use_attr")
418-
get_node(uses, "namespace_module")
419+
get_node(uses, "namespace_module.store")
419420

420421

421-
def test_issue127_attr_write_falls_back_to_module():
422-
"""Writing ``store.flag = value`` should also count as coupling to
423-
the namespace-style module — same fallback rule, applied through
422+
def test_issue127_unresolved_attr_write_emits_edge_to_binding():
423+
"""Writing ``store.flag = value`` should also produce the
424+
binding-level edge — same mechanism, applied through
424425
``set_attribute`` / the Attribute-in-Store path."""
425426
v = _issue127_visitor("namespace_module.py", "consumer.py")
426427
uses = get_in_dict(v.uses_edges, "consumer.write_attr")
427-
get_node(uses, "namespace_module")
428+
get_node(uses, "namespace_module.store")
428429

429430

430431
def test_issue127_chained_access_climbs_to_defined_ancestor():
431-
"""``store.foo.bar`` — neither ``foo`` nor ``bar`` is statically known,
432-
and ``store`` itself is an undefined IMPORTEDITEM. The fallback should
433-
climb past every undefined intermediate and emit exactly one edge to
434-
the nearest defined ancestor: the ``namespace_module`` module.
435-
No edges should point at undefined synthetic ATTRIBUTE nodes — those
436-
are invisible in the rendered graph and just noise in ``uses_edges``."""
432+
"""``store.foo.bar`` — neither ``foo`` nor ``bar`` is statically known.
433+
The intermediate access ``store.foo`` produces an undefined synthetic
434+
ATTRIBUTE Node, and the outer ``.bar`` access has to climb past it to
435+
reach a defined ancestor. After the prequisite to #129, the climb
436+
terminates at ``namespace_module.store`` (the binding's NAME Node)
437+
rather than at the enclosing module. No edges should point at the
438+
undefined ATTRIBUTE intermediates — those are invisible in the
439+
rendered graph and just noise in ``uses_edges``."""
437440
v = _issue127_visitor("namespace_module.py", "chained_consumer.py")
438441
uses = get_in_dict(v.uses_edges, "chained_consumer.use_chained")
439-
get_node(uses, "namespace_module")
442+
get_node(uses, "namespace_module.store")
440443
# No edges to undefined non-wildcard nodes (e.g. namespace_module.store.foo).
441444
for n in uses:
442445
assert n.defined or n.namespace is None, (

0 commit comments

Comments
 (0)