Skip to content

Commit 83f8d43

Browse files
Technologicatclaude
andcommitted
Track iterator protocol methods for for/async for/comprehensions (D8)
- Add _add_iterator_protocol_edges(): __iter__/__next__ for sync, __aiter__/__anext__ for async iteration - visit_For: resolve iterable, add sync protocol edges - visit_AsyncFor: no longer a trivial alias; adds async protocol edges - analyze_comprehension: protocol edges for outermost and inner generators, using each generator's is_async flag - Document double-visit pattern consistently across visit_For, visit_AsyncFor, analyze_comprehension, and _visit_with - Three new tests: for_iter_protocol, async_for_iter_protocol, comprehension_iter_protocol - Update CHANGELOG, TODO_DEFERRED, REMAINING-ITEMS Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent a5868f5 commit 83f8d43

6 files changed

Lines changed: 111 additions & 6 deletions

File tree

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@
99
- **`del` statement protocol tracking**`del obj.attr` now generates a uses
1010
edge to `__delattr__`, and `del obj[key]` to `__delitem__`. Complements the
1111
existing `__enter__`/`__exit__` tracking for `with`.
12+
- **Iterator protocol tracking**`for` loops, `async for` loops, and
13+
comprehensions now generate uses edges to `__iter__`/`__next__` (or
14+
`__aiter__`/`__anext__` for async). Comprehension generators respect
15+
the `is_async` flag.
1216

1317
### Bug fixes
1418

REMAINING-ITEMS.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ v2.0.0 released 2026-02-19.
1515

1616
| Item | Description |
1717
|------|-------------|
18-
| D8 | Iterator protocol tracking + `is_async` |
1918
| D10 | `visit_Name` local variable noise |
2019

2120
### Large
@@ -30,7 +29,7 @@ v2.0.0 released 2026-02-19.
3029

3130
## Completed
3231

33-
All M, S, and N1–N7 items. D1–D7, D9, D11, D16. CI, badges, flake8.
32+
All M, S, and N1–N8, D9, D11, D16. CI, badges, flake8.
3433
See `CHANGELOG.md` for the 2.0.0 manifest.
3534

3635
## Canonical detail

TODO_DEFERRED.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313

1414
- **D6: README: document `--module-level` mode**: ✓ Done (`831f31c`). Also added --svg/--html to modvis CLI.
1515
- **D7: `Del` context tracking**: ✓ Done. `visit_Delete` tracks `__delattr__` for `del obj.attr` and `__delitem__` for `del obj[key]`. Bare `del name` is a no-op (just unbinds). Three tests added.
16-
- **D8: Iterator protocol tracking + `is_async`**: We already track the context manager protocol (`__enter__`/`__exit__` for `with`). Tracking the iterator protocol (`__iter__`/`__next__`) would be a natural addition — and would make `analyze_comprehension`'s ignored `ast.comprehension.is_async` field relevant too (`__aiter__`/`__anext__` vs `__iter__`/`__next__`).
16+
- **D8: Iterator protocol tracking + `is_async`**: ✓ Done. `_add_iterator_protocol_edges()` adds `__iter__`/`__next__` (or `__aiter__`/`__anext__` when async) for `for`, `async for`, and comprehension generators. The `is_async` field on `ast.comprehension` is now used. Three tests added.
1717
- **D9: modvis `filename_to_module_name` cwd fragility**: ✓ Done (`c9cc075`+`a310477`). Added `root` parameter to `filename_to_module_name`, `ImportVisitor`, `create_modulegraph`, and CLI `--root`. Root is inferred by default (walk up past `__init__.py` dirs).
1818
- **D10: `visit_Name` local variable noise**: When a local has no known value, a wildcard `UNKNOWN`-flavored node is created (analyzer.py:721–725). The existing TODO suggests skipping node creation for locals in the innermost scope — would reduce graph noise and postprocessor cleanup work.
1919
- **D11: Plain-text output**: ✓ Done (`d2a5b6a`). `TextWriter` added to `writers.py`; `--text` CLI flag and `"text"` format for both call-graph and module-level modes. Old commented-out plaintext code removed from modvis.

pyan/analyzer.py

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -845,9 +845,33 @@ def visit_AugAssign(self, node):
845845
# consequences in the expand_unknowns() step, if the same name is
846846
# in use elsewhere.)
847847
#
848+
def _add_iterator_protocol_edges(self, iter_node, is_async=False):
849+
"""Add uses edges for the iterator protocol on `iter_node`.
850+
851+
Sync iteration: ``__iter__`` + ``__next__``
852+
Async iteration: ``__aiter__`` + ``__anext__``
853+
"""
854+
if isinstance(iter_node, Node):
855+
from_node = self.get_node_of_current_namespace()
856+
if is_async:
857+
methods = ("__aiter__", "__anext__")
858+
else:
859+
methods = ("__iter__", "__next__")
860+
for methodname in methods:
861+
to_node = self.get_node(iter_node.get_name(), methodname, None, flavor=Flavor.METHOD)
862+
self.logger.debug("Use from %s to %s (iteration)" % (from_node, to_node))
863+
if self.add_uses_edge(from_node, to_node):
864+
self.logger.info("New edge added for Use from %s to %s (iteration)" % (from_node, to_node))
865+
848866
def visit_For(self, node):
849867
self.logger.debug("For-loop, %s:%s" % (self.filename, node.lineno))
850868

869+
# Visit the iterable to resolve the object node, then add protocol edges.
870+
# NOTE: node.iter is visited again inside analyze_binding(); the double
871+
# visit is harmless (resolves to the same node, edges deduplicate).
872+
iter_node = self.visit(node.iter)
873+
self._add_iterator_protocol_edges(iter_node)
874+
851875
targets = canonize_exprs(node.target)
852876
values = canonize_exprs(node.iter)
853877
self.analyze_binding(targets, values)
@@ -858,7 +882,21 @@ def visit_For(self, node):
858882
self.visit(stmt)
859883

860884
def visit_AsyncFor(self, node):
861-
self.visit_For(node) # TODO: alias for now; tag async for in output in a future version?
885+
self.logger.debug("AsyncFor-loop, %s:%s" % (self.filename, node.lineno))
886+
887+
# NOTE: node.iter is visited again inside analyze_binding(); the double
888+
# visit is harmless (resolves to the same node, edges deduplicate).
889+
iter_node = self.visit(node.iter)
890+
self._add_iterator_protocol_edges(iter_node, is_async=True)
891+
892+
targets = canonize_exprs(node.target)
893+
values = canonize_exprs(node.iter)
894+
self.analyze_binding(targets, values)
895+
896+
for stmt in node.body:
897+
self.visit(stmt)
898+
for stmt in node.orelse:
899+
self.visit(stmt)
862900

863901
def visit_ListComp(self, node):
864902
self.logger.debug("ListComp, %s:%s" % (self.filename, node.lineno))
@@ -904,6 +942,7 @@ def analyze_comprehension(self, node, label, field1="elt", field2=None):
904942
iter_node = None
905943
for expr in outermost_iters:
906944
iter_node = self.visit(expr)
945+
self._add_iterator_protocol_edges(iter_node, is_async=outermost.is_async)
907946

908947
# Ensure comprehension scope exists. On Python 3.12+ (PEP 709),
909948
# symtable no longer reports listcomp/setcomp/dictcomp as child scopes.
@@ -924,11 +963,15 @@ def analyze_comprehension(self, node, label, field1="elt", field2=None):
924963
for expr in outermost.ifs:
925964
self.visit(expr)
926965

927-
# TODO: there's also an is_async field we might want to use in a future version of Pyan.
928966
for gen in moregens:
929967
targets = canonize_exprs(gen.target)
930968
values = canonize_exprs(gen.iter)
931969
self.analyze_binding(targets, values)
970+
# Add iterator protocol edges for inner generators.
971+
# NOTE: gen.iter is visited again (already visited inside
972+
# analyze_binding); harmless — same node, edges deduplicate.
973+
inner_iter_node = self.visit(gen.iter)
974+
self._add_iterator_protocol_edges(inner_iter_node, is_async=gen.is_async)
932975
for expr in gen.ifs:
933976
self.visit(expr)
934977

@@ -996,7 +1039,8 @@ def add_uses_enter_exit_of(graph_node):
9961039
expr = withitem.context_expr
9971040
vars = withitem.optional_vars
9981041

999-
# XXX: we currently visit expr twice (again in analyze_binding()) if vars is not None
1042+
# NOTE: expr is visited again inside analyze_binding() when vars is not None;
1043+
# the double visit is harmless (resolves to the same node, edges deduplicate).
10001044
cm_node = self.visit(expr)
10011045
add_uses_enter_exit_of(cm_node)
10021046

tests/test_code/features.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,3 +233,38 @@ def remove_item(registry):
233233
def unbind_local():
234234
tmp = 1
235235
del tmp
236+
237+
238+
# --- Iterator protocol ---
239+
240+
class Sequence:
241+
def __iter__(self):
242+
return self
243+
244+
def __next__(self):
245+
raise StopIteration
246+
247+
248+
def iterate_sequence():
249+
seq = Sequence()
250+
for item in seq: # noqa: F841 # test fixture
251+
pass
252+
253+
254+
class AsyncStream:
255+
def __aiter__(self):
256+
return self
257+
258+
async def __anext__(self):
259+
raise StopAsyncIteration
260+
261+
262+
async def iterate_async_stream():
263+
stream = AsyncStream()
264+
async for chunk in stream: # noqa: F841 # test fixture
265+
pass
266+
267+
268+
def comprehend_sequence():
269+
seq = Sequence()
270+
return [x for x in seq]

tests/test_features.py

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,29 @@ def test_del_name_no_protocol_edge(v):
218218
assert f"{PREFIX}.unbind_local" not in names
219219

220220

221+
# --- Iterator protocol ---
222+
223+
def test_for_iter_protocol(v):
224+
"""for item in seq: creates uses edges to Sequence.__iter__ and __next__."""
225+
uses = get_in_dict(v.uses_edges, f"{PREFIX}.iterate_sequence")
226+
get_node(uses, f"{PREFIX}.Sequence.__iter__")
227+
get_node(uses, f"{PREFIX}.Sequence.__next__")
228+
229+
230+
def test_async_for_iter_protocol(v):
231+
"""async for chunk in stream: creates uses edges to AsyncStream.__aiter__ and __anext__."""
232+
uses = get_in_dict(v.uses_edges, f"{PREFIX}.iterate_async_stream")
233+
get_node(uses, f"{PREFIX}.AsyncStream.__aiter__")
234+
get_node(uses, f"{PREFIX}.AsyncStream.__anext__")
235+
236+
237+
def test_comprehension_iter_protocol(v):
238+
"""[x for x in seq] creates uses edges to Sequence.__iter__ and __next__."""
239+
uses = get_in_dict(v.uses_edges, f"{PREFIX}.comprehend_sequence")
240+
get_node(uses, f"{PREFIX}.Sequence.__iter__")
241+
get_node(uses, f"{PREFIX}.Sequence.__next__")
242+
243+
221244
# --- Type aliases (PEP 695, Python 3.12+) ---
222245

223246
PREFIX_312 = "test_code_312.type_aliases"

0 commit comments

Comments
 (0)