Skip to content

Commit 833c17d

Browse files
committed
Make readme leaner.
1 parent b867c28 commit 833c17d

File tree

1 file changed

+5
-67
lines changed

1 file changed

+5
-67
lines changed

README.md

+5-67
Original file line numberDiff line numberDiff line change
@@ -15,75 +15,13 @@ ______________________________________________________________________
1515
Parallelize the execution of tasks with `pytask-parallel`, a plugin for
1616
[pytask](https://github.com/pytask-dev/pytask).
1717

18-
## Installation
19-
20-
pytask-parallel is available on [PyPI](https://pypi.org/project/pytask-parallel) and
21-
[Anaconda.org](https://anaconda.org/conda-forge/pytask-parallel). Install it with
22-
23-
```console
24-
$ pip install pytask-parallel
25-
26-
# or
27-
28-
$ conda install -c conda-forge pytask-parallel
29-
```
30-
31-
By default, the plugin uses loky's reusable executor.
32-
33-
The following backends are available:
34-
35-
- loky's [`get_reusable_executor`](https://loky.readthedocs.io/en/stable/API.html#loky.get_reusable_executor)
36-
- `ProcessPoolExecutor` or `ThreadPoolExecutor` from
37-
[concurrent.futures](https://docs.python.org/3/library/concurrent.futures.html)
38-
- dask's [`ClientExecutor`](https://distributed.dask.org/en/stable/api.html#distributed.Client.get_executor) allows in combination with [coiled](https://docs.coiled.io/user_guide/index.html) to spawn clusters and workers on AWS, GCP, and other providers with minimal configuration.
39-
40-
## Usage
41-
42-
To parallelize your tasks across many workers, pass an integer greater than 1 or
43-
`'auto'` to the command-line interface.
44-
45-
```console
46-
$ pytask -n 2
47-
$ pytask --n-workers 2
48-
49-
# Starts os.cpu_count() - 1 workers.
50-
$ pytask -n auto
51-
```
52-
53-
Using processes to parallelize the execution of tasks is useful for CPU-bound tasks such
54-
as numerical computations. ([Here](https://stackoverflow.com/a/868577/7523785) is an
55-
explanation of what CPU- or IO-bound means.)
56-
57-
For IO-bound tasks, tasks where the limiting factor is network latency and access to
58-
files, you can parallelize via threads.
59-
60-
```console
61-
pytask --parallel-backend threads
62-
```
63-
64-
You can also set the options in a `pyproject.toml`.
65-
66-
```toml
67-
# This is the default configuration. Note that, parallelization is turned off.
68-
69-
[tool.pytask.ini_options]
70-
n_workers = 1
71-
parallel_backend = "loky" # or processes or threads
72-
```
73-
74-
## Parallelization and Debugging
75-
76-
It is not possible to combine parallelization with debugging. That is why `--pdb` or
77-
`--trace` deactivate parallelization.
78-
79-
If you parallelize the execution of your tasks using two or more workers, do not use
80-
`breakpoint()` or `import pdb; pdb.set_trace()` since both will cause exceptions.
81-
8218
## Documentation
8319

84-
You find the documentation at <https://pytask-parallel.readthedocs.io/en/stable>.
20+
You find the documentation at <https://pytask-parallel.readthedocs.io> with a
21+
[quickstart guide](https://pytask-parallel.readthedocs.io/en/latest/quickstart.html).
8522

8623
## Changes
8724

88-
Consult the [release notes](https://pytask-parallel.readthedocs.io/en/stable/changes.html) to
89-
find out about what is new.
25+
Consult the
26+
[release notes](https://pytask-parallel.readthedocs.io/en/latest/changes.html) to find
27+
out about what is new.

0 commit comments

Comments
 (0)