Capture warnings. (#263)

Author: tobiasraabe

.pre-commit-config.yaml

@@ -117,10 +117,6 @@ repos:
     -   id: check-manifest
         args: [--no-build-isolation]
         additional_dependencies: [setuptools-scm, toml]
--   repo: https://github.com/guilatrova/tryceratops
-    rev: v1.0.1
-    hooks:
-    -   id: tryceratops
 -   repo: https://github.com/pre-commit/mirrors-mypy
     rev: 'v0.942'
     hooks:

docs/source/_static/images/warning.svg

@@ -7,8 +7,10 @@ releases are available on [PyPI](https://pypi.org/project/pytask) and
 
 ## 0.2.1 - 2022-xx-xx
 
-- {pull}`261` adds a config file option to sort entries in live table
 - {pull}`259` adds an `.svg` for profiling tasks.
+- {pull}`261` adds a config file option to sort entries in live table
+- {pull}`262` allows pytask to capture warnings. Here is the
+  [guide](https://pytask-dev.readthedocs.io/en/stable/how_to_guides/capture_warnings.html).
 
 ## 0.2.0 - 2022-04-14
 

docs/source/_static/images/write-a-task.svg

@@ -0,0 +1,130 @@
+# Capture warnings
+
+pytask captures warnings during the execution.
+
+Here is an example with the most infamous warning in the world of scientific Python.
+
+```python
+import pandas as pd
+import pytask
+
+
+def _create_df():
+    df = pd.DataFrame({"a": range(10), "b": range(10, 20)})
+    df[df["a"] < 5]["b"] = 1
+    return df
+
+
+@pytask.mark.products("df.pkl")
+def task_warning(produces):
+    df = _create_df()
+    df.to_pickle(produces)
+```
+
+Running pytask produces
+
+```{image} /_static/images/warning.svg
+```
+
+## Controlling warnings
+
+You can use the `filterwarnings` option in `pyproject.toml` to configure pytasks
+behavior to warnings. For example, the configuration below will ignore all user warnings
+and specific deprecation warnings matching a regex, but will transform all other
+warnings into errors.
+
+```toml
+[tool.pytask.ini_options]
+filterwarnings = [
+    "error",
+    "ignore::UserWarning",
+    # note the use of single quote below to denote "raw" strings in TOML
+    'ignore:function ham\(\) is deprecated:DeprecationWarning',
+]
+```
+
+When a warning matches more than one option in the list, the action for the last
+matching option is performed.
+
+## `@pytask.mark.filterwarnings`
+
+You can use the `@pytask.mark.filterwarnings` to add warning filters to specific test
+items, allowing you to have finer control of which warnings should be captured at test,
+class or even module level:
+
+```python
+import pandas as pd
+import pytask
+
+
+def _create_df():
+    df = pd.DataFrame({"a": range(10), "b": range(10, 20)})
+    df[df["a"] < 5]["b"] = 1
+    return df
+
+
+@pytask.mark.filterwarnings("ignore:.*:SettingWithCopyWarning")
+@pytask.mark.products("df.pkl")
+def task_warning(produces):
+    df = _create_df()
+    df.to_pickle(produces)
+```
+
+Filters applied using a mark take precedence over filters passed on the command line or
+configured by the `filterwarnings` configuration option.
+
+## Disabling warnings summary
+
+Although not recommended, you can use the `--disable-warnings` command-line option to
+suppress the warning summary entirely from the test run output.
+
+## `DeprecationWarning` and `PendingDeprecationWarning`
+
+By default pytask will display `DeprecationWarning` and `PendingDeprecationWarning`
+warnings from user code and third-party libraries. This helps users keep their code
+modern and avoid breakages when deprecated warnings are effectively removed.
+
+Sometimes it is useful to hide some specific deprecation warnings that happen in code
+that you have no control over (such as third-party libraries), in which case you might
+use the warning filters options (ini or marks) to ignore those warnings.
+
+For example:
+
+```toml
+[tool.pytask.ini_options]
+filterwarnings  = [
+    "ignore:.*U.*mode is deprecated:DeprecationWarning"
+]
+```
+
+This will ignore all warnings of type `DeprecationWarning` where the start of the
+message matches the regular expression `".*U.*mode is deprecated"`.
+
+## Debugging warnings
+
+Sometimes it is not clear which line of code triggered a warning. To find the location,
+you can turn warnings into exceptions and then use the {option}`pytask build --pdb` flag
+to enter the debugger.
+
+You can use the configuration to convert warnings to errors by setting
+
+```toml
+[tool.pytask.ini_options]
+filterwarnings = ["error:.*"]
+```
+
+and then run `pytask`.
+
+Or, you use a temporary environment variable. Here is an example for bash
+
+```console
+$ PYTHONWARNINGS=error pytask --pdb
+```
+
+and here for Powershell
+
+```console
+$ $env:PYTHONWARNINGS = 'error'
+$ pytask
+$ Remove-Item env:\PYTHONWARNINGS
+```

docs/source/changes.md

@@ -12,6 +12,7 @@ specific tasks with pytask.
 maxdepth: 1
 ---
 invoking_pytask_extended
+capture_warnings
 repeating_tasks_with_different_inputs_the_pytest_way
 how_to_influence_build_order
 how_to_write_a_plugin

docs/source/how_to_guides/capture_warnings.md

@@ -0,0 +1,25 @@
+from __future__ import annotations
+
+import pandas as pd
+import pytask
+from click.testing import CliRunner
+
+
+def _create_df():
+    df = pd.DataFrame({"a": range(10), "b": range(10, 20)})
+    df[df["a"] < 5]["b"] = 1
+    return df
+
+
+@pytask.mark.produces("df.pkl")
+def task_warning(produces):
+    df = _create_df()
+    df.to_pickle(produces)
+
+
+if __name__ == "__main__":
+    runner = CliRunner()
+
+    pytask.console.record = True
+    runner.invoke(pytask.cli, [__file__])
+    pytask.console.save_svg("warning.svg", title="pytask")

docs/source/how_to_guides/index.md

@@ -68,6 +68,7 @@ def pytask_add_hooks(pm: pluggy.PluginManager) -> None:
     from _pytask import resolve_dependencies
     from _pytask import skipping
     from _pytask import task
+    from _pytask import warnings
 
     pm.register(build)
     pm.register(capture)
@@ -89,6 +90,7 @@ def pytask_add_hooks(pm: pluggy.PluginManager) -> None:
     pm.register(resolve_dependencies)
     pm.register(skipping)
     pm.register(task)
+    pm.register(warnings)
 
 
 @click.group(

scripts/svgs/task_warning.py

@@ -10,7 +10,7 @@
 import attr
 import networkx as nx
 from _pytask.outcomes import ExitCode
-
+from _pytask.warnings_utils import WarningReport
 
 # Location was moved from pluggy v0.13.1 to v1.0.0.
 try:
@@ -62,6 +62,7 @@ class Session:
     scheduler = attr.ib(default=None, type=Any)
     should_stop = attr.ib(default=False, type=Optional[bool])
     """Optional[bool]: Indicates whether the session should be stopped."""
+    warnings = attr.ib(factory=list, type=List[WarningReport])
 
     @classmethod
     def from_config(cls, config: dict[str, Any]) -> Session: