1
1
# pyctdev: python packaging common tasks for developers
2
2
3
3
Tools (and documentation) to support common tasks across many similar
4
- projects, focusing on those making up [ PyViz .org] ( http://pyviz .org ) .
4
+ projects, focusing on those making up [ HoloViz .org] ( http://holoviz .org ) .
5
5
6
6
** Note: documentation is draft/currently being written**
7
7
@@ -20,7 +20,7 @@ ecosystem_setup Common pip setup
20
20
env_capture Report all information required to recreate current environment.
21
21
env_create TODO: create named environment if it doesn't already exist.
22
22
env_export TODO
23
- list_envs
23
+ list_envs
24
24
package_build Build pip package, then install and test all_quick (or other
25
25
package_upload Upload pip packages to pypi
26
26
test_all Run "all" tests
@@ -35,7 +35,7 @@ ecosystem_setup Common conda setup (must be run in base env).
35
35
env_capture Report all information required to recreate current conda environment
36
36
env_create Create named environment if it doesn't already exist
37
37
env_export Generate a pinned environment.yaml from specified env, filtering against specified groups of deps.
38
- list_envs
38
+ list_envs
39
39
miniconda_download Download Miniconda3-latest
40
40
miniconda_install Install Miniconda3-latest to location if not already present
41
41
package_build Build and then test conda.recipe/ (or specified alternative).
@@ -52,13 +52,13 @@ so that people can run commands independently without installing
52
52
doit+pyctdev. This means pyctdev can be viewed as:
53
53
54
54
* documentation of what all the common tasks are
55
-
55
+
56
56
* documentation of the commands necessary to perform those tasks
57
57
58
58
* a way to expose gaps in underlying tools that we might like to
59
59
fill (or exposes our lack of knowledge of how to use them, so we
60
60
can be corrected :) )
61
-
61
+
62
62
* a way to map relatively unchanging "high level tasks" (e.g. "run
63
63
the unit tests") to underlying specific commands that might change over
64
64
time (e.g. as the python packaging ecosystem changes) or that vary
@@ -67,7 +67,7 @@ doit+pyctdev. This means pyctdev can be viewed as:
67
67
* our current best understanding of how to perform the various tasks
68
68
(balancing the best possible way with what's practically possible
69
69
in general, given what tools are currently widely available).
70
-
70
+
71
71
The accompanying [ background] ( background.md ) document (even more draft
72
72
than this one!) contains more details, along with explanations for
73
73
choices. It's broken into the same sections, so can be read alongside
@@ -110,7 +110,7 @@ configuration files). (And once any python's available, doit can be
110
110
used to install other pythons if necessary - currently miniconda and
111
111
anaconda.) Having the same command to run on each platform helps ensure
112
112
that testing, package building, and related tasks are done consistently
113
- across platforms, which is particularly important when developers use
113
+ across platforms, which is particularly important when developers use
114
114
one platform but users will download packages for another.
115
115
116
116
Other suggested tools used by pyctdev are also cross platform: tox,
@@ -124,7 +124,7 @@ conda ecosystem. E.g. `doit develop_install` will typically run `pip
124
124
install -e .[ tests] `, which installs the dependencies using pip and
125
125
then does an editable install. Alternatively, `doit ecosystem=conda
126
126
develop install` will install dependencies using conda, followed by an
127
- editable install. Projects can set a default ecosystem.
127
+ editable install. Projects can set a default ecosystem.
128
128
129
129
The ability to install with pip or conda, create reproducible/isolated
130
130
environments with python tools (virtualenv+pip, or pipenv) or with
@@ -150,13 +150,13 @@ widely supported by both python and conda tools. The dependencies
150
150
listed in setup.py are used for:
151
151
152
152
* end-user pip packages
153
-
153
+
154
154
* end-user conda packages
155
-
155
+
156
156
* developers using conda
157
-
157
+
158
158
* developers using pip
159
-
159
+
160
160
* generating environment files (e.g. examples environment.yml)
161
161
162
162
The abstract dependencies may be transformed to more concrete ones,
@@ -244,16 +244,16 @@ package metadata only once. Currently this is in setup.py. Templating
244
244
is then used for conda build. This prevents the common situation where
245
245
descriptions, URLs, licenses, etc, are mismatched.
246
246
247
- pyctdev expects project is being released first on pypi and on an
247
+ pyctdev expects project is being released first on pypi and on an
248
248
anaconda.org channel. From these sources, conda-forge can be updated, followed by
249
249
anaconda defaults (but we are not necessarily the maintainers of those
250
250
channels).
251
251
252
252
pyctdev is currently primarily supporting pure Python packages. While
253
253
they may often have complex, platform specific dependencies, the
254
- packages controlled by pyct are so far all pure Python. Therefore
254
+ packages controlled by pyct are so far all pure Python. Therefore
255
255
we build noarch: python conda packages where possible. If we start
256
- maintaining packages with binary code, pyct will be extended
256
+ maintaining packages with binary code, pyct will be extended
257
257
to support platform-specific packages, but for now none of our
258
258
packages require that.
259
259
@@ -263,7 +263,7 @@ packages require that.
263
263
For python/pip: typically just pypi.org. But other 'channels' can be
264
264
specified. E.g. test.pypi.org, or a private server.
265
265
266
- The conda packages we maintain for PyViz .org can usually be installed
266
+ The conda packages we maintain for HoloViz .org can usually be installed
267
267
on top of either anaconda defaults or conda-forge. We currently put
268
268
them on anaconda.org pyviz (releases) and pyviz/label/dev (dev
269
269
versions), and only our specific packages are on this channel. For a
@@ -280,7 +280,7 @@ conda-forge and defaults should be updated.
280
280
### 11. How to structure project
281
281
282
282
Although it's not necessary, a common structure simplifies things
283
- across multiple similar projects. Pyviz projects typically have
283
+ across multiple similar projects. HoloViz projects typically have
284
284
repositories that look like:
285
285
286
286
```
@@ -318,7 +318,7 @@ projects, and how? Would rather not have a config file for pyctdev...)
318
318
### 13. What's tested, and how.
319
319
320
320
There are various tools for running tests (e.g. pytest, nose). An aim
321
- of pyctdev is for our pyviz projects to all end up using the same
321
+ of pyctdev is for our HoloViz projects to all end up using the same
322
322
developer tools where possible. And to configure those tools in the
323
323
same kind of way.
324
324
@@ -329,9 +329,9 @@ same kind of way.
329
329
* examples:
330
330
331
331
* notebooks run without error: pytest plugin nbsmoke
332
-
332
+
333
333
* notebooks flakes: pytest plugin nbsmoke
334
-
334
+
335
335
* notebooks "data tests": pytest plugin nbva
336
336
337
337
* performance/benchmark tests: (pytest-benchmark, custom stuff,
@@ -367,7 +367,7 @@ needs to know version (`__init__.py`; packaging: `setup.py`,
367
367
368
368
Storing in one place, and it being the tag rather than in the git repo
369
369
source code, makes it easier to automate various other 'release time'
370
- tasks. Our pyviz projects generally use
370
+ tasks. Our projects generally use
371
371
[ autover] ( https://github.com/pyviz/autover ) (via param).
372
372
373
373
Versioning scheme:
0 commit comments