warning.formatwarning cannot be modified within pytest

[michue-work]

• a detailed description of the bug or problem you are having

I tried to apply a custom format to some warnings generated during tests using the method described in a "Python module of the week"-post. This requires overriding the warnings.formatwarning method as is intended according to the package documentation.

Somehow this approach didn't work at all (custom formatting was not applied). During further investigation, I also tried to modify warnings.showwarning, which finally worked in changing the formatting.
BUT this now breaks pytest's detection of warnings (the warning with modified warnings.showwarning is not counted and doesn't show up in the warnings summary).

• output of pip list from the virtual environment you are using

Package    Version
---------- -------
colorama   0.4.6  
iniconfig  2.0.0  
packaging  24.2   
pip        25.0   
pluggy     1.5.0  
pytest     8.3.4  
ruff       0.9.3  
setuptools 65.5.0

• pytest and operating system versions

pytest 8.3.4
Python 3.11.9
Windows 10 Enterprise (Version 22H2, Build 19045.5371)

• minimal example if possible

import sys
import warnings


def formatwarning_without_trace(message, category, filename, lineno, line=None) -> str:
    category = category.__name__
    return f'{filename}:{lineno}: {category}: {message}\n'


# same as original, but with custom formatwarning
def showwarning_without_trace(message, category, filename, lineno, file=None, line=None):  # noqa: PLR0913
    if file is None:
        file = sys.stderr
        if file is None:
            # sys.stderr is None when run with pythonw.exe:
            # warnings get lost
            return
    text = formatwarning_without_trace(message, category, filename, lineno, None)
    try:  # noqa: SIM105
        file.write(text)
    except OSError:
        # the file (probably stderr) is invalid - this warning gets lost.
        pass

    warnings.warn(message=message, category=category)


# warning shows normally, is counted / appears in the summary
def test_warning_warn():
    warnings.warn('standard warning.warn')


# warning is counted / appears in the summary, but doesn't use the custom formatting
def test_warning_formatwarning():
    warnings_formatwarning = warnings.formatwarning
    warnings.formatwarning = showwarning_without_trace

    warnings.warn('custom warning.formatwarning')

    warnings.formatwarning = warnings_formatwarning


# warning uses custom formatting, but is not counted / is missing in the summary
def test_warning_showwarning():
    warnings_showwarning = warnings.showwarning
    warnings.showwarning = showwarning_without_trace

    warnings.warn('custom warning.showwarning')

    warnings.showwarning = warnings_showwarning

produces

============================= test session starts =============================
platform win32 -- Python 3.11.9, pytest-8.3.4, pluggy-1.5.0 -- [...]\pytest-warnings\venv\Scripts\python.exe
cachedir: .pytest_cache
rootdir: [...]\pytest-warnings
configfile: pyproject.toml
collecting ... collected 3 items

test_warnings.py::test_warning_warn PASSED                               [ 33%]
test_warnings.py::test_warning_formatwarning PASSED                      [ 66%]
test_warnings.py::test_warning_showwarning PASSED                        [100%]

============================== warnings summary ===============================
test_warnings.py::test_warning_warn
  [...]\pytest-warnings\test_warnings.py:28: UserWarning: standard warning.warn
    warnings.warn('standard warning.warn')

test_warnings.py::test_warning_formatwarning
  [...]\test_warnings.py:35: UserWarning: custom warning.formatwarning
    warnings.warn('custom warning.formatwarning')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================== 3 passed, 2 warnings in 0.04s ========================
Finished running

[The-Compiler]

Overriding formatwarning doesn't work because pytest doesn't format the warnings during the test, it stores them as "raw" data and only calls formatwarning in the pytest_warning_recorded hook. If you correctly override formatwarning around that hook (perhaps with a hookwrapper), that should work just fine.

As for overriding showwarning, this is basically equivalent to:

import warnings

with warnings.catch_warnings(record=True) as recorded:
    warnings.warn("this gets recorded")
print(recorded)


with warnings.catch_warnings(record=True) as lost:
    warnings.showwarning = lambda *args: None   # your custom implementation
    warnings.warn("this gets lost")
print(lost)

so there isn't really anything that pytest could do to make that work.

All in all, I don't think there is anything actionable for pytest there.