Async generator/anext with default-tuple-value results in SystemError: <class 'StopIteration'> returned with exception set

[manuel-koch]

Bug report

Bug description:

I asked a question on stackoverflow because I found a strange error while writing code that makes use an async generator function and the built-in anext() function when using a tuple value for the default argument of anext().

One of the stackoverflow answers contains a good analysis and a possible cause of the problem.

The failing code is the second line with anext():

import asyncio

async def generator(it=None):
    if it is not None:
        yield (it, it)

async def my_func():
    # results in a=1 b=1
    a, b = await anext(generator(1), (2, 3))
    
    # results in no printing
    async for a, b in generator():
        print(a, b)
    
    # raises exception
    a, b = await anext(generator(), (2, 3))

loop = asyncio.new_event_loop()
loop.run_until_complete(my_func())

It raises this unexpected exception:

StopAsyncIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/manuel/.pyenv/versions/3.12.0/lib/python3.12/asyncio/base_events.py", line 664, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "<stdin>", line 6, in my_func
SystemError: <class 'StopIteration'> returned a result with an exception set

CPython versions tested on:

3.11, 3.12, 3.13

Operating systems tested on:

macOS

Linked PRs

• gh-128078: Clear exception in anext before calling _PyGen_SetStopIterationValue #128780
• [3.12] gh-128078: Clear exception in anext before calling _PyGen_SetStopIterationValue (GH-128780) #128784
• [3.13] gh-128078: Clear exception in anext before calling _PyGen_SetStopIterationValue (GH-128780) #128785
• gh-128078: Simplify logic in _PyGen_SetStopIterationValue (follow-up to gh-128780) #128287
• [3.13] gh-128078: Use PyErr_SetRaisedException in _PyGen_SetStopIterationValue (GH-128287) #128789
• [3.12] gh-128078: Use PyErr_SetRaisedException in _PyGen_SetStopIterationValue (GH-128287) #128790

[kumaraditya303]

FWIW I think its a bug in implementation of anext.
Specifically in these lines:

cpython/Objects/iterobject.c

Lines 386 to 388 in 3a570c6

	if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) {
	_PyGen_SetStopIterationValue(obj->default_value);
	}

This should clear the PyExc_StopAsyncIteration before calling _PyGen_SetStopIterationValue.

The following patch fixes it for me and all async gen tests pass:

diff --git a/Objects/iterobject.c b/Objects/iterobject.c
index 135ced9ea1f..b0938690dca 100644
--- a/Objects/iterobject.c
+++ b/Objects/iterobject.c
@@ -384,6 +384,7 @@ anextawaitable_iternext(anextawaitableobject *obj)
         return result;
     }
     if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) {
+        PyErr_Clear();
         _PyGen_SetStopIterationValue(obj->default_value);
     }
     return NULL;

[picnixz]

This is something I've also wondered. We could clear the exception before calling _PyGen_SetStopIterationValue but Mark also suggested removing the specialized path for tuples. So we can perhaps do both in my PR?

[kumaraditya303]

I suggest to do both in different PRs, I would prefer to backport the simpler fix.

[picnixz]

Doing both is fine. I'm not on my dev session anymore but if you want to do the PyErr_Clear() part, it's better to merge it first. Then I'll merge mine where I can just check that no exception is currently set instead.