Understand the changes of event-loop when moving from 0.21

[jefflieu]

Hello,
We've been stuck with version 0.21 for a while and now got some time to move on and update to latest version.
I think we've got it working but not quite understand the concept of the changes behind it. What's the changes in the event-loop mechanism before and after version 0.21

Before:

@pytest.fixture(scope="session")
def event_loop():
     try:
         event_loop = asyncio.get_running_loop()
     except RuntimeError:
         event_loop = asyncio.new_event_loop()
     yield event_loop
     event_loop.close()

After:

from pytest_asyncio import is_async_test
def pytest_collection_modifyitems(items):
    pytest_asyncio_tests = (item for item in items if is_async_test(item))
    session_scope_marker = pytest.mark.asyncio(loop_scope="session")
    for async_test in pytest_asyncio_tests:
        async_test.add_marker(session_scope_marker, append=False)

And

[tool.pytest.ini_options]
asyncio_default_fixture_loop_scope = "session"

I understand that these set the the event loop for the test and the fixture to the same "session" loop.
I did some readings and discussion,
but I still don't understand exactly what's the changes that broke it and requires this setting. The changes doesn't seem to be explicitly documented anywhere, it may be embedded in discussions that I might have missed.
I can see documents describing a migration path, but not explain the mechanics of the changes behind the scene.

Without pytest_collection_modifyitems .. I do some experiment and it shows me that background tasks that I launch in the setup of the fixture doesn't run when the test function itself starts.

For example i have this file

import asyncio
import pytest
import pytest_asyncio
import logging


def fork_task(coro):
    # loop = asyncio.get_event_loop()
    loop = asyncio.get_running_loop()
    return loop.create_task(coro)


async def background_task():
    for i in range(10):
        logging.info(f"i={i}")
        await asyncio.sleep(1)


@pytest_asyncio.fixture(scope="package")
async def test_fixture():
    fork_task(background_task())
    yield None


@pytest.mark.asyncio
async def test_queue_writing(test_fixture):
    logging.info("Test starts")
    await asyncio.sleep(10)
    logging.info("Test ends")

Without pytest_collection_modifyitems or (@pytest.mark.asyncio(loop_scope="session")) --> the background_task doesn't run when it enters the test_queue_writing

Could you please point me back to the basic of pytest-asyncio?
Thank you very much!

[seifertm]

Could you please point me back to the basic of pytest-asyncio?

@jefflieu The concepts section of the pytest-asyncio documentation explains the idea of having event loops with different scopes. We're always open for suggestions to improve the documentation. Is there specific information that you're missing?

Which pytest-asyncio version are you upgrading to?

Side note about this piece of code:

from pytest_asyncio import is_async_test
def pytest_collection_modifyitems(items):
    pytest_asyncio_tests = (item for item in items if is_async_test(item))
    session_scope_marker = pytest.mark.asyncio(loop_scope="session")
    for async_test in pytest_asyncio_tests:
        async_test.add_marker(session_scope_marker, append=False)

In recent versions of pytest-asyncio, you can replace this section of the conftest.py with a simple setting.

[jefflieu]

Hi,
I think what i'm missing is the changes between version v0.21 and v1.1 that require me to have:

[tool.pytest.ini_options]
asyncio_default_fixture_loop_scope = "session"

from pytest_asyncio import is_async_test
def pytest_collection_modifyitems(items):
    pytest_asyncio_tests = (item for item in items if is_async_test(item))
    session_scope_marker = pytest.mark.asyncio(loop_scope="session")
    for async_test in pytest_asyncio_tests:
        async_test.add_marker(session_scope_marker, append=False)

Is it correct to say that

• in v0.21, everything is running with 1 loop.
• Now with version 1.1, we have multiple loops and that requires the new settings?

(I think part of my problem is that I never run an asyncio with multiple event loops)

[seifertm]

Is it correct to say that

* in v0.21, everything is running with 1 loop.

* Now with version 1.1, we have multiple loops and that requires the new settings?

It can be accurate for specific test suites, but the statement is not generally correct.

TL;DR: If you only use a single event loop with session scope, set asyncio-default-fixture-loop-scope and asyncio_default_test_loop_scope to session.

Longer explanation:
In v0.21, the default is to have one event loop per test function by default. Let's assume you want to change the default loop scope to module and you put the following override into a conftest.py that affects all your tests:

@pytest.fixture(scope="module")
def event_loop():
    …

This change will change the loop scope to module for all tests. However, if your test suite is organized in multiple directories and you put the same code into a conftest.py in a subfolder of your test suite, the override will only affect part of your tests. Only the tests affected by the conftest.py (i.e. the tests in the subfolder) will use an event loop with module scope. All other tests will still use event loops with function scope. That means pytest-asyncio v0.21 did also support multiple loops with different scopes. Please refer to the pytest docs for further details on conftest.py.

We just established that in pytest-asyncio v0.21 the scope of an event loop used by a test was indirectly defined by the contents of the nearest conftest.py. The event loop scope of a test In pytest-asyncio v1.0, however, can be defined explicitly via @pytest.mark.asyncio(loop_scope=…) for tests and @pytest_asyncio.fixture(loop_scope=…) for fixtures.
The configuration options asyncio_default_test_loop_scope and asyncio_default_fixture_loop_scope are merely shortcuts that allows users to provide a global default for all fixtures and tests.

Does that answer you question?

[jefflieu]

Hi,
thank you! @seifertm , yes, that partially answered my questions.
I'm on the same page regarding what you described.

Could you please also clarify these?
Regarding the loop of the fixtures, in version 0.21,

• Was the async fixtures's loop also indirectly defined by "event_loop" in the nearest conftest.py?
• Although there were multiple loops, but in a particular run: all fixtures and tests run with the same "event_loop" whether that is module-scoped or function-scoped. Is this statement correct?

In version v1.0, from the concept page, I think I understand it as follows:

• Things are different, fixtures and test functions could run in totally 2 different loops. For example, fixtures could be run with session's loop and test functions can run with function's loop. These are set by the "loopscope".

So, if these are all correct I think this is the critical difference between the 2 versions.

• v0.21: we only have 1 single loop for both fixtures and tests which is the "event_loop" fixture. This is defined by the "event_loop" as a fixture.
• v1.0: we could have many event loops running at the same time

That leads to another question regarding "multiple event loops" running at the same time in the example I posted.

import asyncio
import pytest
import pytest_asyncio
import logging


def fork_task(coro):
    # loop = asyncio.get_event_loop()
    loop = asyncio.get_running_loop()
    return loop.create_task(coro)


async def background_task():
    for i in range(10):
        logging.info(f"i={i}")
        await asyncio.sleep(1)


@pytest_asyncio.fixture(scope="package")
async def test_fixture():
    fork_task(background_task())
    yield None


@pytest.mark.asyncio
async def test_queue_writing(test_fixture):
    logging.info("Test starts")
    await asyncio.sleep(10)
    logging.info("Test ends")

If I really want fixtures and tests running on different loops, what am I missing to make the example work?

Thank you very much! Really appreciate you following and correcting my understanding.

Jeff

[seifertm]

• Was the async fixtures's loop also indirectly defined by "event_loop" in the nearest conftest.py?

• Although there were multiple loops, but in a particular run: all fixtures and tests run with the same "event_loop" whether that is module-scoped or function-scoped. Is this statement correct?

Yes, fixtures and tests were run in the same loop. With recent versions of pytest-asyncio, modifying the loop scope of either tests or fixtures means that the user has to make sure that fixtures and tests run in the same loop. However, there are plans to make this easier in the future (see #1095 )

• session's loop and test functions can run with function's loop. These are set by the "loopscope".

So, if these are all correct I think this is the critical difference between the 2 versions.

* v0.21: we only have 1 single loop for both fixtures and tests which is the "event_loop" fixture. This is defined by the "event_loop" as a fixture.

* v1.0: we could have many event loops running at the same time

Setting aside the fact that there could also be different event_loop fixture definitions in v0.21 (see my previous comment), I think we have the same understanding. It depends on your definition of "at the same time". Tests in pytest run single-threaded and sequentially. There can only be a single async event loop per thread. But it's totally possible that the first test in a module uses a different event loop that another test, yes.

If I really want fixtures and tests running on different loops, what am I missing to make the example work?

It depends on your setting of asyncio_default_test_loop_scope and asyncio_default_fixture_loop_scope. Assuming you use the (deprecated): To make your example work, use @pytest_asyncio.fixture(scope="package", loop_scope="function"). This will create a single background_task per package, but run it the same event loop as test_queue_writing.

[jefflieu]

Thank you. That answered all my questions! Really appreciate it!

[jefflieu]

any taker on this? are my questions valid or totally not making any sense (because I don't understand something) ?.

[jefflieu]

Sorry but your suggestion of having a fixture of scope package but loop_scope="function" doesn't work
It gives error:

ScopeMismatch: You tried to access the function scoped fixture _function_scoped_runner with a package scoped request object. Requesting fixture stack:
test_pytest_asyncio.py:19:  def test_fixture()
Requested fixture:
../venv/lib64/python3.10/site-packages/pytest_asyncio/plugin.py:748:  def _scoped_runner(event_loop_policy) -> 'Iterator[Runner]'
============================================================================================= short test summary info ==============================================================================================
ERROR test_pytest_asyncio.py::test_queue_writing - Failed: ScopeMismatch: You tried to access the function scoped fixture _function_scoped_runner with a package scoped request object. Requesting fixture stack:

My test code:

import asyncio
import pytest
import pytest_asyncio
import logging


def fork_task(coro):
    # loop = asyncio.get_event_loop()
    loop = asyncio.get_running_loop()
    return loop.create_task(coro)


async def background_task():
    for i in range(10):
        logging.info(f"i={i}")
        await asyncio.sleep(1)


@pytest_asyncio.fixture(scope="package", loop_scope="function")
async def test_fixture():
    fork_task(background_task())
    yield None


@pytest.mark.asyncio(loop_scope="function")
async def test_queue_writing(test_fixture):
    logging.info("Test starts")
    await asyncio.sleep(10)
    logging.info("Test ends")

---

I'm a bit confused. I totally understand the concept of scope of fixture from pytest doc.
But what does it mean for a fixture to have a loop_scope that is different from the scope of itself?

[seifertm]

Sorry, I forgot about this limitation.

The scope is the regular pytest scope that you're familiar with. I'll refer to it as caching scope. The loop_scope refers to the scope of the asyncio event loop. In other words, the loop scope decides when the event loop is reset.

The loop scope can only be the same or a larger scope than the caching scope. It's impossible to cache the result of test_fixture for the whole test package (as specified by scope="package"), if the loop is reset for every function (as specified by loop_scope="function").

As a consequence, you either need to widen the loop scope of your fixture and test to package or you need to narrow the caching scope to function.

We should introduce a more descriptive error message for this case.
I also think this deserves a dedicated section it the pytest-asyncio docs.

[jefflieu]

OK, it makes sense now.
I think I understand it now and what we're trying to achieve.
In summary, in a particular "test run" --> all fixtures and test function must use the same loop. For example in a run you have X fixtures and Y tests, the loop scope of all of those must be the same and is the widest scope of the fixtures.
Thank you!
(I still feel like using the old way of "event_loop" in nearest conftest.py would do this more explicitly though)