@@ -11,15 +11,15 @@ You find a tutorial on type hints {doc}`here <../type_hints>`.
11
11
12
12
If you want to avoid type annotations for now, look at the tab named ` produces ` .
13
13
14
- The ` Decorators ` tab documents the deprecated approach that should not be used
15
- anymore and will be removed in version v0.5.
14
+ The ` Decorators ` tab documents the deprecated approach that should not be used anymore
15
+ and will be removed in version v0.5.
16
16
17
17
In this tutorial, we only deal with local files. If you want to use pytask with files
18
18
online, S3, GCP, Azure, etc., read the
19
19
{doc}` guide on remote files <../how_to_guides/remote_files> ` .
20
20
21
- First, we focus on defining products that should already be familiar to you. Then,
22
- we focus on how you can declare task dependencies.
21
+ First, we focus on defining products that should already be familiar to you. Then, we
22
+ focus on how you can declare task dependencies.
23
23
24
24
We use the same project as before and add a ` task_plot_data.py ` module.
25
25
@@ -47,9 +47,9 @@ my_project
47
47
Let's revisit the task from the {doc}` previous tutorial <write_a_task> ` that we defined
48
48
in ` task_data_preparation.py ` .
49
49
50
- :::: {tab-set}
50
+ ````` {tab-set}
51
51
52
- ::: {tab-item} Python 3.10+
52
+ ```` {tab-item} Python 3.10+
53
53
:sync: python310plus
54
54
55
55
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_products_py310.py
@@ -59,9 +59,9 @@ in `task_data_preparation.py`.
59
59
{class}`~pytask.Product` allows marking an argument as a product. After the
60
60
task has finished, pytask will check whether the file exists.
61
61
62
- :::
62
+ ````
63
63
64
- ::: {tab-item} Python 3.8+
64
+ ```` {tab-item} Python 3.8+
65
65
:sync: python38plus
66
66
67
67
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_products_py38.py
@@ -71,9 +71,9 @@ task has finished, pytask will check whether the file exists.
71
71
{class}`~pytask.Product` allows marking an argument as a product. After the
72
72
task has finished, pytask will check whether the file exists.
73
73
74
- :::
74
+ ````
75
75
76
- ::: {tab-item} &# 8203 ; ` produces `
76
+ ```` {tab-item} prodouces
77
77
:sync: produces
78
78
79
79
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_products_produces.py
@@ -84,9 +84,9 @@ Tasks can use `produces` as a "magic" argument name. Every value, or in this cas
84
84
passed to this argument is automatically treated as a task product. Here, we pass the
85
85
path as the default argument.
86
86
87
- :::
87
+ ````
88
88
89
- ::: {tab-item} Decorators
89
+ ```` {tab-item} Decorators
90
90
:sync: decorators
91
91
92
92
```{warning}
@@ -103,13 +103,15 @@ task. After the task has finished, pytask will check whether the file exists.
103
103
Add `produces` as an argument of the task function to get access to the same path inside
104
104
the task function.
105
105
106
- :::
107
- ::::
106
+ ````
108
107
109
- :::{tip}
110
- If you do not know about {mod}` pathlib ` check out [ ^ id3 ] and [ ^ id4 ] . The module is
111
- beneficial for handling paths conveniently and across platforms.
112
- :::
108
+ `````
109
+
110
+ ``` {tip}
111
+ If you do not know about {mod}`pathlib` check out this guide by
112
+ [RealPython](https://realpython.com/python-pathlib/). The module is beneficial for
113
+ handling paths conveniently and across platforms.
114
+ ```
113
115
114
116
## Dependencies
115
117
@@ -119,24 +121,24 @@ To show how dependencies work, we extend our project with another task that plot
119
121
data generated with ` task_create_random_data ` . The task is called ` task_plot_data ` , and
120
122
we will define it in ` task_plot_data.py ` .
121
123
122
- :::: {tab-set}
124
+ ````` {tab-set}
123
125
124
- ::: {tab-item} Python 3.10+
126
+ ```` {tab-item} Python 3.10+
125
127
:sync: python310plus
126
128
127
129
To specify that the task relies on the data set `data.pkl`, you can add the path
128
130
to the function signature while choosing any argument name, here `path_to_data`.
129
131
130
- pytask assumes that all function arguments that do not have a {class}` ~pytask. ` Product`
132
+ pytask assumes that all function arguments that do not have a {class}`~pytask.Product`
131
133
annotation are dependencies of the task.
132
134
133
135
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_dependencies_py310.py
134
136
:emphasize-lines: 11
135
137
```
136
138
137
- :::
139
+ ````
138
140
139
- ::: {tab-item} Python 3.8+
141
+ ```` {tab-item} Python 3.8+
140
142
:sync: python38plus
141
143
142
144
To specify that the task relies on the data set `data.pkl`, you can add the path
@@ -149,9 +151,9 @@ annotation are dependencies of the task.
149
151
:emphasize-lines: 11
150
152
```
151
153
152
- :::
154
+ ````
153
155
154
- ::: {tab-item} &# 8203 ; ` produces `
156
+ ```` {tab-item} prodouces
155
157
:sync: produces
156
158
157
159
To specify that the task relies on the data set `data.pkl`, you can add the path to the
@@ -164,9 +166,9 @@ pytask assumes that all function arguments that are not passed to the argument
164
166
:emphasize-lines: 9
165
167
```
166
168
167
- :::
169
+ ````
168
170
169
- ::: {tab-item} Decorators
171
+ ```` {tab-item} Decorators
170
172
:sync: decorators
171
173
172
174
```{warning}
@@ -182,8 +184,8 @@ access the dependency path inside the function and load the data.
182
184
:emphasize-lines: 9, 11
183
185
```
184
186
185
- :::
186
- ::::
187
+ ````
188
+ `````
187
189
188
190
Now, let us execute the two paths.
189
191
@@ -195,36 +197,36 @@ Now, let us execute the two paths.
195
197
Dependencies and products do not have to be absolute paths. If paths are relative, they
196
198
are assumed to point to a location relative to the task module.
197
199
198
- :::: {tab-set}
200
+ ````` {tab-set}
199
201
200
- ::: {tab-item} Python 3.10+
202
+ ```` {tab-item} Python 3.10+
201
203
:sync: python310plus
202
204
203
205
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_relative_py310.py
204
206
:emphasize-lines: 8
205
207
```
206
208
207
- :::
209
+ ````
208
210
209
- ::: {tab-item} Python 3.8+
211
+ ```` {tab-item} Python 3.8+
210
212
:sync: python38plus
211
213
212
214
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_relative_py38.py
213
215
:emphasize-lines: 8
214
216
```
215
217
216
- :::
218
+ ````
217
219
218
- ::: {tab-item} &# 8203 ; ` produces `
220
+ ```` {tab-item} prodouces
219
221
:sync: produces
220
222
221
223
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_relative_produces.py
222
224
:emphasize-lines: 4
223
225
```
224
226
225
- :::
227
+ ````
226
228
227
- ::: {tab-item} Decorators
229
+ ```` {tab-item} Decorators
228
230
:sync: decorators
229
231
230
232
```{warning}
@@ -241,16 +243,16 @@ You can also use absolute and relative paths as strings that obey the same rules
241
243
If you use `depends_on` or `produces` as arguments for the task function, you will have
242
244
access to the paths of the targets as {class}`pathlib.Path`.
243
245
244
- :::
245
- ::::
246
+ ````
247
+ `````
246
248
247
249
## Multiple dependencies and products
248
250
249
251
Of course, tasks can have multiple dependencies and products.
250
252
251
- :::: {tab-set}
253
+ ````` {tab-set}
252
254
253
- ::: {tab-item} Python 3.10+
255
+ ```` {tab-item} Python 3.10+
254
256
:sync: python310plus
255
257
256
258
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_multiple1_py310.py
@@ -263,9 +265,9 @@ structures if needed.
263
265
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_multiple2_py310.py
264
266
```
265
267
266
- :::
268
+ ````
267
269
268
- ::: {tab-item} Python 3.8+
270
+ ```` {tab-item} Python 3.8+
269
271
:sync: python38plus
270
272
271
273
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_multiple1_py38.py
@@ -278,9 +280,9 @@ structures if needed.
278
280
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_multiple2_py38.py
279
281
```
280
282
281
- :::
283
+ ````
282
284
283
- ::: {tab-item} &# 8203 ; ` produces `
285
+ ```` {tab-item} prodouces
284
286
:sync: produces
285
287
286
288
If your task has multiple products, group them in one container like a dictionary
@@ -294,9 +296,9 @@ You can do the same with dependencies.
294
296
```{literalinclude} ../../../docs_src/tutorials/defining_dependencies_products_multiple2_produces.py
295
297
```
296
298
297
- :::
299
+ ````
298
300
299
- ::: {tab-item} Decorators
301
+ ```` {tab-item} Decorators
300
302
:sync: decorators
301
303
302
304
```{warning}
@@ -405,8 +407,8 @@ def task_fit_model(depends_on, produces):
405
407
}
406
408
```
407
409
408
- :::
409
- ::::
410
+ ````
411
+ `````
410
412
411
413
(after)=
412
414
@@ -431,8 +433,7 @@ def task_plot_data(...):
431
433
You can also pass a list of task functions.
432
434
433
435
The second mode is to pass an expression, a substring of the name of the dependent
434
- tasks. Here, we can pass the function name or a significant part of the function
435
- name.
436
+ tasks. Here, we can pass the function name or a significant part of the function name.
436
437
437
438
``` python
438
439
@task (after = " random_data" )
@@ -450,9 +451,3 @@ You will learn more about expressions in {doc}`selecting_tasks`.
450
451
- An overview of all ways to specify dependencies and products and their strengths and
451
452
weaknesses can be found in
452
453
{doc}` ../how_to_guides/interfaces_for_dependencies_products ` .
453
-
454
- ## References
455
-
456
- [ ^ id3 ] : The official documentation for {mod}` pathlib ` .
457
-
458
- [ ^ id4 ] : A guide for pathlib by [ RealPython] ( https://realpython.com/python-pathlib/ ) .
0 commit comments