fix(types): type hints from future python versions

[InvincibleRMC]

Description

As shown in #5678 some types need to use typing_extensions if they don't exists yet. Similar to #5663 but, for the rest of the types.

Suggested changelog entry:

• Add typing_extensions alternatives for all types that need them

[timohl]

Great! Those match all stubgen errors concerning new Python features I found while working on #5678.

There is one additional typing issue I found though:

pybind11/include/pybind11/cast.h

Lines 1450 to 1453 in 1dd85ef

	template <>
	struct handle_type_name<weakref> {
	static constexpr auto name = const_name("weakref");
	};

weakref is not the correct type but just the module name.
I think this should be weakref.ReferenceType (docs)
Maybe we can squeeze this in here as well?

Your solution for X | Y looks good to me as well.

For testing I am not so sure about all those if sys.version_info >= (3, XX): with double asserts.
Would it maybe make sense to somehow move that into a common place as well (like in conftest.py?

[InvincibleRMC]

For testing I am not so sure about all those if sys.version_info >= (3, XX): with double asserts. Would it maybe make sense to somehow move that into a common place as well (like in conftest.py?

I forgot that mypy can use future python features in .pyi files would it better/simpler to just switch everything over to X | Y in the test cases?

[timohl]

For testing I am not so sure about all those if sys.version_info >= (3, XX): with double asserts. Would it maybe make sense to somehow move that into a common place as well (like in conftest.py?

I forgot that mypy can use future python features in .pyi files would it better/simpler to just switch everything over to X | Y in the test cases?

That would make the tests a lot simpler, so I think this is a good idea.
With across_version_type_hint_checker everything should look cleaner.

How is the test output when the assert inside across_version_type_hint_checker fails?
Is it still clear where the error occurs and what the context is (just curious how pytest handles asserts in helper functions)?

I can have a deeper look tomorrow evening on your latest changes.

[henryiii]

Please, never import from conftest.py. You can put fixtures, etc. there, but don't import from it. It causes all sorts of issues (such as the issue holding up cibuildwheel 3.0 in numpy right now!); it's not meant to be an importable module for users. We sadly have one case (tests/test_docs_advanced_cast_custom.py), but I'd like to get rid of that, not add more. Edit: it's only at type check time!

Instead of adding a test/utils directory just for this, and requiring PyTest 7+, this can be wrapped in a fixture for now.

Since we do have one case of importing from conftest, I'm also okay to leave this in for now and add the test utils in a followup later, as anything broken by it will already have workarounds.

[henryiii]

Does this also reduce the quality of assertion rewriting?

[henryiii]

FYI, wrapping it in a fixture could look like:

@pytest.fixture
def backport_typehints() -> Callable[[str], str]:
    d = {}
    if sys.version_info < (3, 13):
        d["typing.TypeIs"] = "typing_extensions.TypeIs"
        d["types.CapsuleType"] = "typing_extensions.CapsuleType"
    if sys.version_info < (3, 12):
        d["collections.abc.Buffer"] = "typing_extensions.Buffer"
    if sys.version_info < (3, 11):
        d["typing.Never"] = "typing_extensions.Never"
    if sys.version_info < (3, 10):
        d["typing.TypeGuard"] = "typing_extensions.TypeGuard"
       
    def backport(text: str) -> str:
        for old, new in d.items():
            text = text.replace(old, new)
        return text
    return backport

[timohl]

Please, never import from conftest.py. You can put fixtures, etc. there, but don't import from it. It causes all sorts of issues (such as the issue holding up cibuildwheel 3.0 in numpy right now!); it's not meant to be an importable module for users. We sadly have one case (tests/test_docs_advanced_cast_custom.py), but I'd like to get rid of that, not add more. Edit: it's only at type check time!

Just to clarify and to learn from it (since I added that conftest.py import in tests/test_docs_advanced_cast_custom.py):
Would you also discourage from importing conftest.py while using if TYPE_CHECKING?
Generally, I like having type hints for function arguments for better code readability and navigation.
Vscode/Pylance is capable of matching test arguments to fixtures and show gray type hints without manual type annotations, so I could see not having those in tests.

[timohl]

Two minor comments, otherwise this looks good to me.