Merge branch 'main' into delayed-nodes

Author: tobiasraabe

.github/workflows/main.yml

@@ -53,7 +53,7 @@ jobs:
         run: tox -e test -- -m "unit or (not integration and not end_to_end)" --cov=src --cov=tests --cov-report=xml -n auto
 
       - name: Upload unit test coverage reports to Codecov with GitHub Action
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         with:
           flags: unit
 
@@ -62,7 +62,7 @@ jobs:
         run: tox -e test -- -m integration --cov=src --cov=tests --cov-report=xml -n auto
 
       - name: Upload integration test coverage reports to Codecov with GitHub Action
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         with:
           flags: integration
 
@@ -71,6 +71,6 @@ jobs:
         run: tox -e test -- -m end_to_end --cov=src --cov=tests --cov-report=xml -n auto
 
       - name: Upload end_to_end test coverage reports to Codecov with GitHub Action
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
         with:
           flags: end_to_end

.github/workflows/update-plugin-list.yml

@@ -37,7 +37,7 @@ jobs:
         run: python scripts/update_plugin_list.py
 
       - name: Create Pull Request
-        uses: peter-evans/create-pull-request@153407881ec5c347639a548ade7d8ad1d6740e38
+        uses: peter-evans/create-pull-request@a4f52f8033a6168103c2538976c07b467e8163bc
         with:
           commit-message: '[automated] Update plugin list'
           author: 'Tobias Raabe <[email protected]>'

.pre-commit-config.yaml

@@ -24,18 +24,6 @@ repos:
     -   id: python-check-mock-methods
     -   id: python-no-log-warn
     -   id: text-unicode-replacement-char
--   repo: https://github.com/asottile/reorder-python-imports
-    rev: v3.12.0
-    hooks:
-    -   id: reorder-python-imports
-        args: [--py38-plus, --add-import, 'from __future__ import annotations']
-        exclude: ^(docs_src/)
--   repo: https://github.com/asottile/reorder-python-imports
-    rev: v3.12.0
-    hooks:
-    -   id: reorder-python-imports
-        args: [--py38-plus]
-        files: ^(docs_src/)
 # -   repo: https://github.com/tox-dev/pyproject-fmt
 #     rev: 1.2.0
 #     hooks:
@@ -45,7 +33,7 @@ repos:
     hooks:
     -   id: sort-all
 -   repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.2.0
+    rev: v0.3.1
     hooks:
       - id: ruff-format
       - id: ruff
@@ -90,14 +78,14 @@ repos:
         ]
         files: (docs/.)
 -   repo: https://github.com/nbQA-dev/nbQA
-    rev: 1.7.1
+    rev: 1.8.4
     hooks:
     -   id: nbqa-black
     -   id: nbqa-isort
     -   id: nbqa-mypy
         args: [--ignore-missing-imports]
 -   repo: https://github.com/kynan/nbstripout
-    rev: 0.6.2
+    rev: 0.7.1
     hooks:
     -   id: nbstripout
         exclude: (docs)

docs/source/changes.md

@@ -17,6 +17,9 @@ releases are available on [PyPI](https://pypi.org/project/pytask) and
   relative to the config file.
 - {pull}`555` uses new-style hook wrappers and requires pluggy 1.3 for typing.
 - {pull}`557` fixes an issue with `@task(after=...)` in notebooks and terminals.
+- {pull}`566` makes universal-pathlib an official dependency.
+- {pull}`568` restricts `task_files` to a list of patterns and raises a better error.
+- {pull}`569` removes the hooks related to the creation of the DAG.
 
 ## 0.4.5 - 2024-01-09
 

docs/source/conf.py

@@ -4,6 +4,7 @@
 documentation: https://www.sphinx-doc.org/en/master/usage/configuration.html
 
 """
+
 from __future__ import annotations
 
 import inspect

docs/source/how_to_guides/remote_files.md

@@ -7,16 +7,11 @@ lots of use cases to deal with remote files.
   get started. So, some tasks reference remote files instead of local files.
 - You store the workflow results in remote storage to save and distribute them.
 
-pytask uses [universal_pathlib](https://github.com/fsspec/universal_pathlib) to work
+pytask uses [universal-pathlib](https://github.com/fsspec/universal_pathlib) to work
 with remote files. The package provides a {mod}`pathlib`-like interface, making it very
 easy to interact with files from an HTTP(S)-, Dropbox-, S3-, GCP-, Azure-based
 filesystem, and many more.
 
-```{warning}
-universal_pathlib does currently not support Python 3.12. To track progress, check the
-[releases `>0.1.4`](https://github.com/fsspec/universal_pathlib/releases).
-```
-
 ## HTTP(S)-based filesystem
 
 As an example for dealing with an HTTP(S)-based filesystem, we will download the iris

docs/source/reference_guides/configuration.md

@@ -242,7 +242,7 @@ strict_markers = true
 Change the pattern which identify task files.
 
 ```toml
-task_files = "task_*.py"  # default
+task_files = ["task_*.py"]  # default
 
 task_files = ["task_*.py", "tasks_*.py"]
 ```

docs/source/reference_guides/hookspecs.md

@@ -62,25 +62,6 @@ The following hooks traverse directories and collect tasks from files.
 .. autofunction:: pytask_collect_log
 ```
 
-## Resolving Dependencies
-
-The following hooks are designed to build a DAG from tasks and dependencies and check
-which files have changed and need to be re-run.
-
-```{warning}
-This step is still experimental and likely to change in the future. If you are planning
-to write a plugin which extends pytask in this dimension, please, start a discussion
-before writing a plugin. It may make your life easier if changes in pytask anticipate
-your plugin.
-```
-
-```{eval-rst}
-.. autofunction:: pytask_dag
-.. autofunction:: pytask_dag_create_dag
-.. autofunction:: pytask_dag_log
-
-```
-
 ## Execution
 
 The following hooks execute the tasks and log information on the result in the terminal.

docs/source/tutorials/defining_dependencies_products.md

@@ -72,7 +72,7 @@ task has finished, pytask will check whether the file exists.
 
 ````
 
-````{tab-item} prodouces
+````{tab-item} produces
 :sync: produces
 
 ```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_products_produces.py
@@ -132,7 +132,7 @@ annotation are dependencies of the task.
 
 ````
 
-````{tab-item} prodouces
+````{tab-item} produces
 :sync: produces
 
 To specify that the task relies on the data set `data.pkl`, you can add the path to the
@@ -178,7 +178,7 @@ are assumed to point to a location relative to the task module.
 
 ````
 
-````{tab-item} prodouces
+````{tab-item} produces
 :sync: produces
 
 ```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_relative_produces.py

docs/source/tutorials/making_tasks_persist.md

@@ -13,7 +13,7 @@ in the database such that the subsequent execution will skip the task successful
 
 ## When is this useful?
 
-- You ran a formatter like Black on the files in your project and want to prevent the
+- You ran a formatter like ruff on the files in your project and want to prevent the
   longest-running tasks from being rerun.
 - You extend a repetition of a task function but do not want to rerun all tasks.
 - You want to integrate a task that you have already run elsewhere. Copy over the