Unable to determine which files to ship inside the wheel on docker build when following doc

[flpStrri]

When following the documented deployment "For a Hatch project" I was hit by the following error every time I tried to build my docker with the RUN hatch env prune && hatch env create production before the COPY package_name /app/package_name.

When I make it otherwise it builds. We need to make some updates to this documentation section or I am messing up something big?

ValueError: Unable to determine which files to ship inside the wheel using the following heuristics: https://hatch.pypa.io/latest/plugins/builder/wheel/#default-file-selection

At least one file selection option must be defined in the `tool.hatch.build.targets.wheel` table, see: https://hatch.pypa.io/latest/config/build/

As an example, if you intend to ship a directory named `foo` that resides within a `src` directory located at the root of your project, you can define the following:

[tool.hatch.build.targets.wheel]
packages = ["src/foo"]

[br3ndonland]

Thanks for pointing this out @flpStrri. It sounds like you're seeing two issues:

1. Need to copy source code package into image before running hatch env commands
2. ValueError: Unable to determine which files to ship inside the wheel

I've updated the docs to address these two issues. Happy to consider further updates to the docs if needed.

1. Need to copy source code package into image before running hatch env commands

When I set up a Hatch project exactly as described in the docs, I do not get a ValueError, but a metadata-generation-failed error instead (expand).

~/dev/inboard-example-from-docs via 🐳 orbstack via 🐍 v3.11.7 on ☁️  (us-east-2) took 6s
❯ docker build . -t inboard-example-from-docs:latest --no-cache --platform=linux/amd64 --progress=plain
#0 building with "orbstack" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 418B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for ghcr.io/br3ndonland/inboard:fastapi
#3 DONE 0.0s

#4 [1/5] FROM ghcr.io/br3ndonland/inboard:fastapi
#4 CACHED

#5 [internal] load build context
#5 transferring context: 370B done
#5 DONE 0.0s

#6 [2/5] COPY pyproject.toml README.md /app/
#6 DONE 0.0s

#7 [3/5] WORKDIR /app
#7 DONE 0.0s

#8 [4/5] RUN hatch env prune && hatch env create production
#8 0.285 Removing environment: default
#8 0.711 Creating environment: production
#8 0.830 Installing project
#8 3.898   error: subprocess-exited-with-error
#8 3.898
#8 3.898   × Preparing metadata (pyproject.toml) did not run successfully.
#8 3.898   │ exit code: 1
#8 3.898   ╰─> [26 lines of output]
#8 3.898       Traceback (most recent call last):
#8 3.898         File "/app/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
#8 3.898           main()
#8 3.898         File "/app/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
#8 3.898           json_out['return_val'] = hook(**hook_input['kwargs'])
#8 3.898                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#8 3.898         File "/app/.venv/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
#8 3.898           whl_basename = backend.build_wheel(metadata_directory, config_settings)
#8 3.898                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/build.py", line 58, in build_wheel
#8 3.898           return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard'])))
#8 3.898                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/builders/plugin/interface.py", line 90, in build
#8 3.898           self.metadata.validate_fields()
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/metadata/core.py", line 243, in validate_fields
#8 3.898           _ = self.version
#8 3.898               ^^^^^^^^^^^^
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/metadata/core.py", line 128, in version
#8 3.898           self._version = self._get_version()
#8 3.898                           ^^^^^^^^^^^^^^^^^^^
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/metadata/core.py", line 226, in _get_version
#8 3.898           version = self.hatch.version.cached
#8 3.898                     ^^^^^^^^^^^^^^^^^^^^^^^^^
#8 3.898         File "/tmp/pip-build-env-yhaptkb7/overlay/lib/python3.11/site-packages/hatchling/metadata/core.py", line 1424, in cached
#8 3.898           raise type(e)(message) from None
#8 3.898       OSError: Error getting the version from source `regex`: file does not exist: package_name/__init__.py
#8 3.898       [end of output]
#8 3.898
#8 3.898   note: This error originates from a subprocess, and is likely not a problem with pip.
#8 3.900 error: metadata-generation-failed
#8 3.900
#8 3.900 × Encountered error while generating package metadata.
#8 3.900 ╰─> See above for output.
#8 3.900
#8 3.900 note: This is an issue with the package mentioned above, not pip.
#8 3.900 hint: See above for details.
#8 ERROR: process "/bin/sh -c hatch env prune && hatch env create production" did not complete successfully: exit code: 1
------
 > [4/5] RUN hatch env prune && hatch env create production:
3.898       [end of output]
3.898
3.898   note: This error originates from a subprocess, and is likely not a problem with pip.
3.900 error: metadata-generation-failed
3.900
3.900 × Encountered error while generating package metadata.
3.900 ╰─> See above for output.
3.900
3.900 note: This is an issue with the package mentioned above, not pip.
3.900 hint: See above for details.
------
Dockerfile:9
--------------------
   7 |     COPY pyproject.toml README.md /app/
   8 |     WORKDIR /app
   9 | >>> RUN hatch env prune && hatch env create production
  10 |
  11 |     # Install Python app
--------------------
ERROR: failed to solve: process "/bin/sh -c hatch env prune && hatch env create production" did not complete successfully: exit code: 1

The error is occurring because the pyproject.toml has a dynamic version that is stored in package_name/__init__.py, but the source code in package_name hasn't been copied into the Docker image yet.

The reason the Dockerfile was set up to COPY the source code after installing the environment is explained in the docs:

Organizing the Dockerfile this way helps leverage the Docker build cache. Files and commands that change most frequently are added last to the Dockerfile. Next time the image is built, Docker will skip any layers that didn't change, speeding up builds.

To continue leveraging the Docker build cache while moving the COPY command earlier, COPY --link syntax can be used. This syntax was introduced in Dockerfile 1.4.0, so there is a comment at the top of the Dockerfile examples setting the Dockerfile version requirement (# syntax=docker/dockerfile:1).

I've updated the docs to move the COPY --link step earlier to avoid the error.

2. ValueError: Unable to determine which files to ship inside the wheel

I was not able to reproduce the exact error you reported, but I have a suggestion for resolving it. To resolve the "ValueError: Unable to determine which files to ship inside the wheel" issue you reported, the pyproject.toml may need an additional section for the wheel target. The error may be due to an update in Hatchling 1.9 (Hatchling is the build backend for Hatch). The update added a requirement for additional metadata.

The example previously just had sdist (source distribution) metadata like this:

[tool.hatch.build.targets.sdist]
include = ["/package_name"]

With the additional wheel metadata it looks like this:

[tool.hatch.build.targets.sdist]
include = ["/package_name"]

[tool.hatch.build.targets.wheel]
packages = ["package_name"]

I've updated the docs with the additional wheel metadata.