Skip to content

Create tool that can infer missing test fixtures #15119

@JukkaL

Description

@JukkaL

Coming up with test fixtures (e.g. [builtins ...]) adds a lot of friction, especially for new contributors. Write a tool to automatically infer and add a missing fixture to newly added test cases. It doesn't need to work perfectly, but we can perhaps implement something that works for, say, 80-90% of test cases.

The tool could work like this:

  • Use git commands to determine newly added test cases by comparing against an ancestor master branch commit.
  • For each new test case that doesn't have a fixture yet, look at the code being tested and guess a suitable fixture. For example, if a test case uses tuples, suggest fixtures/tuple.pyi.

The inference might need some pretty ad hoc heuristics (e.g. which fixture to pick if multiple features are used), but that's okay.

This would need to be documented prominently in the developer docs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic-developerIssues relevant to mypy developers

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions