Skip to content

Add --ignore CLI option and ignore_files config field #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 0 additions & 30 deletions LICENSE

This file was deleted.

93 changes: 93 additions & 0 deletions LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
Files: *
Copyright: 2025, Scientific Python Developers
2024, Lars Grüter
License: BSD 3-Clause

-------------------------------------------------------------------------------

Files: src/docstub/_vendored/stdlib.py
Copyright: 2001-2025, Python Software Foundation
License: PSF-2.0
Comment on lines +8 to +10
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanv could I ask you to take a look if I handled the license situation correctly here? I'd like to vendor Python's glob.translate which is only available from 3.13 onward.

I'm also not sure if need to adapt the SPDX license identifier in the pyproject.toml since the vendored part now uses a different one. 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been starting to use the SPDX identifiers. There is now a new PEP that allows you to use a specifier+file in pyproject.toml.

Your file seems to be formatted roughly correctly according to https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/#stanzas, except for the dashes.


-------------------------------------------------------------------------------

License: BSD-3-Clause

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of the vector package developers nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


-------------------------------------------------------------------------------

License: PSF-2.0

PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
--------------------------------------------

1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using this software ("Python") in source or binary form and
its associated documentation.

2. Subject to the terms and conditions of this License Agreement, PSF hereby
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
analyze, test, perform and/or display publicly, prepare derivative works,
distribute, and otherwise use Python alone or in any derivative version,
provided, however, that PSF's License Agreement and PSF's notice of copyright,
i.e., "Copyright (c) 2001 Python Software Foundation; All Rights Reserved"
are retained in Python alone or in any derivative version prepared by Licensee.

3. In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python.

4. PSF is making Python available to Licensee on an "AS IS"
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.

5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.

6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.

7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee. This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.

8. By copying, installing or otherwise using Python, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.
2 changes: 2 additions & 0 deletions docs/command_line.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ Options:
--config PATH Set one or more configuration file(s) explicitly.
Otherwise, it will look for a `pyproject.toml` or
`docstub.toml` in the current directory.
--ignore GLOB Ignore files matching this glob-style pattern. Can be
used multiple times.
--group-errors Group identical errors together and list where they
occurred. Will delay showing errors until all files have
been processed. Otherwise, simply report errors as the
Expand Down
95 changes: 95 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Configuration reference

Docstub will automatically look for configuration in files named

- `pyproject.toml`, and
- `docstub.toml`

in the current working directory.
If config files are explicitly passed to the command line interface via the `--config` option, docstub won't look implicitly look for files in the current directory.
Multiple configuration files can be used, whose content will be merged.

Out of the box, docstub makes use of an internal configuration file [`numpy_config.toml`](../src/docstub/numpy_config.toml) which provides defaults to use NumPy types.


## Configuration fields in `[tool.docstub]`

All configuration must be declared inside a `[tool.docstub]` table.


### `ignore_files`

- [TOML type](https://toml.io/en/latest): array of string(s)

Ignore files and directories matching these [glob-style patterns](https://docs.python.org/3/library/glob.html#glob.translate).
Patterns that don't start with "/" are interpreted as relative to the
directory that contains the Python package for which stubs are generated.

Example:

```toml
[tool.docstub]
ignore_files = [
"**/tests",
]
```

- Will ignore any directory anywhere that is named `tests`.


### `types`

- [TOML type](https://toml.io/en/latest): table, mapping string to string

Types and their external modules to use in docstrings.
Docstub can't yet automatically discover where to import types from other packages from.
Instead, you can provide this information explicitly.
Any type on the left side will be associated with the given "module" on the right side.

Example:

```toml
[tool.docstub.types]
Path = "pathlib"
NDArray = "numpy.typing"
```

- Will allow using `Path` in docstrings and will use `from pathlib import Path` to import the type.
- Will allow using `NDarray` in docstrings and will use `from numpy.typing import NDArray` to import the type.


### `type_prefixes`

- [TOML type](https://toml.io/en/latest): table, mapping string to string

Prefixes for external modules to match types in docstrings.
Docstub can't yet automatically discover where to import types from other packages from.
Instead, you can provide this information explicitly.
Any type in a docstring whose prefix matches the name given on the left side, will be associated with the given "module" on the right side.

Example:

```toml
[tool.docstub.type_prefixes]
np = "numpy"
plt = "matplotlib.pyplot
```

- Will match `np.uint8` and `np.typing.NDarray` and use `import numpy as np`.
- Will match `plt.Figure` use `import matplotlib.pyplot as plt`.


### `type_nicknames`

- [TOML type](https://toml.io/en/latest): table, mapping string to string

Nicknames for types that can be used in docstrings to describe valid Python types or annotations.

Example:

```toml
[tool.docstub.type_nicknames]
func = "Callable"
```

- Will map `func` to the `Callable` type from the `typing` module.
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ maintainers = [
description = "Generate Python stub files from docstrings"
readme = "README.md"
license = "BSD-3-Clause"
license-files = ["LICENSE"]
license-files = ["LICENSE.txt"]
requires-python = ">=3.12"
keywords = ["typing", "stub files", "docstings", "numpydoc"]
classifiers = [
Expand Down Expand Up @@ -120,6 +120,7 @@ run.source = ["docstub"]
Path = "pathlib"

[tool.docstub.type_prefixes]
re = "re"
cst = "libcst"
lark = "lark"
numpydoc = "numpydoc"
Expand Down
67 changes: 42 additions & 25 deletions src/docstub/_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,12 @@
)
from ._cache import FileCache
from ._config import Config
from ._stubs import (
from ._path_utils import (
STUB_HEADER_COMMENT,
Py2StubTransformer,
try_format_stub,
walk_python_package,
walk_source_and_targets,
)
from ._stubs import Py2StubTransformer, try_format_stub
from ._utils import ErrorReporter, GroupedErrorReporter
from ._version import __version__

Expand All @@ -43,22 +42,24 @@ def _load_configuration(config_paths=None):
numpy_config = Config.from_toml(Config.NUMPY_PATH)
config = config.merge(numpy_config)

pyproject_toml = Path.cwd() / "pyproject.toml"
if pyproject_toml.is_file():
logger.info("using %s", pyproject_toml)
add_config = Config.from_toml(pyproject_toml)
config = config.merge(add_config)

docstub_toml = Path.cwd() / "docstub.toml"
if docstub_toml.is_file():
logger.info("using %s", docstub_toml)
add_config = Config.from_toml(docstub_toml)
config = config.merge(add_config)

for path in config_paths:
logger.info("using %s", path)
add_config = Config.from_toml(path)
config = config.merge(add_config)
if config_paths:
for path in config_paths:
logger.info("using %s", path)
add_config = Config.from_toml(path)
config = config.merge(add_config)

else:
pyproject_toml = Path.cwd() / "pyproject.toml"
if pyproject_toml.is_file():
logger.info("using %s", pyproject_toml)
add_config = Config.from_toml(pyproject_toml)
config = config.merge(add_config)

docstub_toml = Path.cwd() / "docstub.toml"
if docstub_toml.is_file():
logger.info("using %s", docstub_toml)
add_config = Config.from_toml(docstub_toml)
config = config.merge(add_config)

return config

Expand All @@ -78,12 +79,17 @@ def _setup_logging(*, verbose):
)


def _collect_types(root_path):
def _collect_types(root_path, *, ignore=()):
"""Collect types.

Parameters
----------
root_path : Path
ignore : Sequence[str], optional
Don't yield files matching these glob-like patterns. The pattern is
interpreted relative to the root of the Python package unless it starts
with "/". See :ref:`glob.translate(..., recursive=True, include_hidden=True)`
for more details on the precise implementation.

Returns
-------
Expand All @@ -98,7 +104,7 @@ def _collect_types(root_path):
name=f"{__version__}/collected_types",
)
if root_path.is_dir():
for source_path in walk_python_package(root_path):
for source_path in walk_python_package(root_path, ignore=ignore):
logger.info("collecting types in %s", source_path)
types_in_source = collect_cached_types(source_path)
types.update(types_in_source)
Expand Down Expand Up @@ -162,6 +168,13 @@ def cli():
"Otherwise, it will look for a `pyproject.toml` or `docstub.toml` in the "
"current directory.",
)
@click.option(
"--ignore",
type=str,
multiple=True,
metavar="GLOB",
help="Ignore files matching this glob-style pattern. Can be used multiple times.",
)
@click.option(
"--group-errors",
is_flag=True,
Expand All @@ -182,7 +195,7 @@ def cli():
@click.option("-v", "--verbose", count=True, help="Print more details (repeatable).")
@click.help_option("-h", "--help")
@report_execution_time()
def run(root_path, out_dir, config_paths, group_errors, allow_errors, verbose):
def run(root_path, out_dir, config_paths, ignore, group_errors, allow_errors, verbose):
"""Generate Python stub files.

Given a `PACKAGE_PATH` to a Python package, generate stub files for it.
Expand All @@ -194,7 +207,8 @@ def run(root_path, out_dir, config_paths, group_errors, allow_errors, verbose):
----------
root_path : Path
out_dir : Path
config_paths : list[Path]
config_paths : Sequence[Path]
ignore : Sequence[str]
group_errors : bool
allow_errors : int
verbose : str
Expand All @@ -212,9 +226,10 @@ def run(root_path, out_dir, config_paths, group_errors, allow_errors, verbose):
)

config = _load_configuration(config_paths)
config = config.merge(Config(ignore_files=list(ignore)))

types = common_known_types()
types |= _collect_types(root_path)
types |= _collect_types(root_path, ignore=config.ignore_files)
types |= {
type_name: KnownImport(import_path=module, import_name=type_name)
for type_name, module in config.types.items()
Expand Down Expand Up @@ -245,7 +260,9 @@ def run(root_path, out_dir, config_paths, group_errors, allow_errors, verbose):

# Stub generation ---------------------------------------------------------

for source_path, stub_path in walk_source_and_targets(root_path, out_dir):
for source_path, stub_path in walk_source_and_targets(
root_path, out_dir, ignore=config.ignore_files
):
if source_path.suffix.lower() == ".pyi":
logger.debug("using existing stub file %s", source_path)
with source_path.open() as fo:
Expand Down
Loading