-
Notifications
You must be signed in to change notification settings - Fork 878
Description
Issue Description
After trying to understand why ADD https://remote/repo.git was not extracting all the files, I noticed it was because they were listed in .dockerignore, so Buildah was removing them. BuildKit on the other hand happily extracts all the files. As far as I understand, .dockerignore is for local files and shouldn't affect remote sources, so BuildKit seems to be correct on this.
Steps to reproduce the issue
Buildah script
#!/bin/sh
podman run --privileged --rm -i quay.io/buildah/stable:latest <<BUILD
cat > .dockerignore <<EOF
*
!docker/*.sh
EOF
cat > Dockerfile <<EOF
FROM alpine
ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
RUN ls -la /podman-tag
EOF
buildah build .
exit
BUILDDocker BuildKit script
#!/bin/sh
CONTAINER="$(podman run --privileged --rm -d docker.io/docker:dind)"
podman exec -i "${CONTAINER}" sh <<BUILD
cat > .dockerignore <<EOF
*
!docker/*.sh
EOF
cat > Dockerfile <<EOF
FROM alpine
ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
RUN ls -la /podman-tag
EOF
until [ -e /var/run/docker.sock ]; do
sleep 1
done
docker buildx build . --progress=plain
exit
BUILD
podman stop "${CONTAINER}"Podman Build script (for completeness)
#!/bin/sh
podman run --privileged --rm -i quay.io/podman/stable:latest <<BUILD
cat > .dockerignore <<EOF
*
!docker/*.sh
EOF
cat > Dockerfile <<EOF
FROM alpine
ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
RUN ls -la /podman-tag
EOF
podman build .
exit
BUILDDescribe the results you received
Buildah output (files missing)
STEP 1/3: FROM alpine
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob sha256:1074353eec0db2c1d81d5af2671e56e00cf5738486f5762609ea33d606f88612
Copying config sha256:e7b39c54cdeca0d2aae83114bb605753a5f5bc511fe8be7590e38f6d9f915dad
Writing manifest to image destination
STEP 2/3: ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
STEP 3/3: RUN ls -la /podman-tag
total 12
drwxr-xr-x 3 root root 4096 Dec 25 08:06 .
dr-xr-xr-x 20 root root 4096 Dec 25 08:06 ..
drwxr-xr-x 2 root root 4096 Dec 25 08:06 docker
COMMIT
Getting image source signatures
Copying blob sha256:7bb20cf5ef67526cb843d264145241ce4dde09a337b5be1be42ba464de9a672d
Copying blob sha256:b3f1beb2f4c48dc313a4bbcff68b1527c21346fcd6d45142f1e904498391547a
Copying config sha256:b2e0f26d736158b992d1c0e6353b01fb81feadacccf0417f26f962ee98fb5222
Writing manifest to image destination
--> b2e0f26d7361
b2e0f26d736158b992d1c0e6353b01fb81feadacccf0417f26f962ee98fb5222
Docker output (all files)
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 136B done
#1 DONE 0.0s
#2 [internal] load metadata for docker.io/library/alpine:latest
#2 DONE 1.9s
#3 [internal] load .dockerignore
#3 transferring context: 55B done
#3 DONE 0.0s
#4 [1/3] FROM docker.io/library/alpine:latest@sha256:865b95f46d98cf867a156fe4a135ad3fe50d2056aa3f25ed31662dff6da4eb62
#4 resolve docker.io/library/alpine:latest@sha256:865b95f46d98cf867a156fe4a135ad3fe50d2056aa3f25ed31662dff6da4eb62 done
#4 sha256:1074353eec0db2c1d81d5af2671e56e00cf5738486f5762609ea33d606f88612 0B / 3.86MB 0.2s
#4 sha256:1074353eec0db2c1d81d5af2671e56e00cf5738486f5762609ea33d606f88612 3.86MB / 3.86MB 0.4s done
#4 extracting sha256:1074353eec0db2c1d81d5af2671e56e00cf5738486f5762609ea33d606f88612 0.0s done
#4 DONE 0.5s
#5 [2/3] ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
#5 0.410 8216e9db9ea7b4d1b8d687fe17dfefc5e053decb refs/tags/v5.0.0
#5 0.415 e71ec6f1d94d2d97fb3afe08aae0d8adaf8bddf0 refs/tags/v5.0.0^{}
#5 0.013 Initialized empty Git repository in /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.overlayfs/snapshots/4/fs/
#5 2.372 From https://github.com/containers/podman
#5 2.372 * [new tag] v5.0.0 -> v5.0.0
#5 2.373 8216e9db9ea7b4d1b8d687fe17dfefc5e053decb
#5 DONE 3.3s
#6 [2/3] ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
#6 DONE 0.8s
#7 [3/3] RUN ls -la /podman-tag
#7 0.121 total 1000
#7 0.121 drwxr-xr-x 21 root root 4096 Dec 25 08:08 .
#7 0.121 drwxr-xr-x 1 root root 4096 Dec 25 08:08 ..
#7 0.121 -rw-r--r-- 1 root root 42980 Dec 25 08:08 .cirrus.yml
#7 0.121 -rw-r--r-- 1 root root 20 Dec 25 08:08 .dockerignore
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 .fmf
#7 0.121 drwxr-xr-x 5 root root 4096 Dec 25 08:08 .github
#7 0.121 -rw-r--r-- 1 root root 694 Dec 25 08:08 .gitignore
#7 0.121 -rw-r--r-- 1 root root 2226 Dec 25 08:08 .golangci.yml
#7 0.121 -rw-r--r-- 1 root root 1631 Dec 25 08:08 .packit.sh
#7 0.121 -rw-r--r-- 1 root root 2358 Dec 25 08:08 .packit.yaml
#7 0.121 -rw-r--r-- 1 root root 937 Dec 25 08:08 .pre-commit-config.yaml
#7 0.121 -rw-r--r-- 1 root root 558 Dec 25 08:08 .readthedocs.yaml
#7 0.121 -rw-r--r-- 1 root root 210 Dec 25 08:08 CODE-OF-CONDUCT.md
#7 0.121 -rw-r--r-- 1 root root 17405 Dec 25 08:08 CONTRIBUTING.md
#7 0.121 -rw-r--r-- 1 root root 3158 Dec 25 08:08 DISTRO_PACKAGE.md
#7 0.121 -rw-r--r-- 1 root root 4345 Dec 25 08:08 DOWNLOADS.md
#7 0.121 -rw-r--r-- 1 root root 2565 Dec 25 08:08 ISSUE-EOL-POLICY.md
#7 0.121 -rw-r--r-- 1 root root 11359 Dec 25 08:08 LICENSE
#7 0.121 -rw-r--r-- 1 root root 40572 Dec 25 08:08 Makefile
#7 0.121 -rw-r--r-- 1 root root 532 Dec 25 08:08 OWNERS
#7 0.121 -rw-r--r-- 1 root root 12124 Dec 25 08:08 README.md
#7 0.121 -rw-r--r-- 1 root root 489583 Dec 25 08:08 RELEASE_NOTES.md
#7 0.121 -rw-r--r-- 1 root root 17750 Dec 25 08:08 RELEASE_PROCESS.md
#7 0.121 -rw-r--r-- 1 root root 235 Dec 25 08:08 SECURITY.md
#7 0.121 -rw-r--r-- 1 root root 1540 Dec 25 08:08 Vagrantfile
#7 0.121 -rw-r--r-- 1 root root 2297 Dec 25 08:08 build_osx.md
#7 0.121 -rw-r--r-- 1 root root 5001 Dec 25 08:08 build_windows.md
#7 0.121 drwxr-xr-x 8 root root 4096 Dec 25 08:08 cmd
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 cni
#7 0.121 -rw-r--r-- 1 root root 21914 Dec 25 08:08 commands-demo.md
#7 0.121 -rw-r--r-- 1 root root 212 Dec 25 08:08 commands.md
#7 0.121 drwxr-xr-x 6 root root 4096 Dec 25 08:08 completions
#7 0.121 drwxr-xr-x 15 root root 4096 Dec 25 08:08 contrib
#7 0.121 drwxr-xr-x 3 root root 4096 Dec 25 08:08 dependencies
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 docker
#7 0.121 drwxr-xr-x 4 root root 4096 Dec 25 08:08 docs
#7 0.121 -rw-r--r-- 1 root root 11096 Dec 25 08:08 go.mod
#7 0.121 -rw-r--r-- 1 root root 81051 Dec 25 08:08 go.sum
#7 0.121 drwxr-xr-x 4 root root 4096 Dec 25 08:08 hack
#7 0.121 -rw-r--r-- 1 root root 545 Dec 25 08:08 install.md
#7 0.121 drwxr-xr-x 11 root root 4096 Dec 25 08:08 libpod
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 logo
#7 0.121 drwxr-xr-x 38 root root 4096 Dec 25 08:08 pkg
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 plans
#7 0.121 -rw-r--r-- 1 root root 169 Dec 25 08:08 pyproject.toml
#7 0.121 -rw-r--r-- 1 root root 5066 Dec 25 08:08 rootless.md
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 rpm
#7 0.121 drwxr-xr-x 20 root root 4096 Dec 25 08:08 test
#7 0.121 -rw-r--r-- 1 root root 14640 Dec 25 08:08 transfer.md
#7 0.121 -rw-r--r-- 1 root root 59327 Dec 25 08:08 troubleshooting.md
#7 0.121 drwxr-xr-x 2 root root 4096 Dec 25 08:08 utils
#7 0.121 drwxr-xr-x 14 root root 4096 Dec 25 08:08 vendor
#7 0.121 drwxr-xr-x 3 root root 4096 Dec 25 08:08 version
#7 0.121 -rw-r--r-- 1 root root 3316 Dec 25 08:08 winmake.ps1
#7 DONE 0.1s
#8 exporting to image
#8 exporting layers
#8 exporting layers 2.1s done
#8 exporting manifest sha256:0ffa2fa39a08a828bf5ae4423b1a79c9af04605cee7170d7727d4542270c4866 done
#8 exporting config sha256:8e127ddca3a90058eacc8983a4c709ceec31f4538139b63a662cbc78000de7c3 done
#8 exporting attestation manifest sha256:e4d9c093acddaffc3080f61eeb14233ee82fe5be4d47ac235a0d5e794f8465ff done
#8 exporting manifest list sha256:e4df19a8e86028517dd81b786eb90e2cf99b4b9792108d92ee63a995e7bc5b0d done
#8 naming to moby-dangling@sha256:e4df19a8e86028517dd81b786eb90e2cf99b4b9792108d92ee63a995e7bc5b0d done
#8 unpacking to moby-dangling@sha256:e4df19a8e86028517dd81b786eb90e2cf99b4b9792108d92ee63a995e7bc5b0d
#8 unpacking to moby-dangling@sha256:e4df19a8e86028517dd81b786eb90e2cf99b4b9792108d92ee63a995e7bc5b0d 1.1s done
#8 DONE 3.2s
e902630bce96a437a46b164fc835868865a300402c2b73d79ff4c009122b61fe
Podman Build output (files missing)
STEP 1/3: FROM alpine
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob sha256:1074353eec0db2c1d81d5af2671e56e00cf5738486f5762609ea33d606f88612
Copying config sha256:e7b39c54cdeca0d2aae83114bb605753a5f5bc511fe8be7590e38f6d9f915dad
Writing manifest to image destination
STEP 2/3: ADD https://github.com/containers/podman.git#v5.0.0 /podman-tag
--> 54643f29da33
STEP 3/3: RUN ls -la /podman-tag
total 12
drwxr-xr-x 3 root root 4096 Dec 25 08:08 .
dr-xr-xr-x 20 root root 4096 Dec 25 08:08 ..
drwxr-xr-x 2 root root 4096 Dec 25 08:08 docker
COMMIT
--> 89512ee30920
89512ee30920b8f95a317cdde49ab6a05b5e9b2060302823138716cd40d758ae
Describe the results you expected
I expected Buildah to add all the contents of https://github.com/containers/podman.git#v5.0.0, regardless of what is in .dockerignore.
buildah version output
I'm able to reproduce this both directly from Arch Linux's buildah and running from quay.io/buildah/stable:
buildah version
$ buildah version
Version: 1.42.2
Go Version: go1.25.5 X:nodwarf5
Image Spec: 1.1.1
Runtime Spec: 1.2.1
CNI Spec: 1.1.0
libcni Version: v1.3.0
image Version: 5.38.0
Git Commit: c0cc97255c76f7a3d87bfdf5e7b97f013f0bfe85
Built: Wed Dec 3 05:00:04 2025
OS/Arch: linux/amd64
BuildPlatform: linux/amd64podman run --privileged --rm -i quay.io/buildah/stable:latest buildah version
$ podman image ls quay.io/buildah/stable:latest
REPOSITORY TAG IMAGE ID CREATED SIZE
quay.io/buildah/stable latest 1590af20ad6f 17 hours ago 555 MB
$ podman run --privileged --rm -i quay.io/buildah/stable:latest buildah version
Version: 1.42.2
Go Version: go1.25.4 X:nodwarf5
Image Spec: 1.1.1
Runtime Spec: 1.2.1
CNI Spec: 1.1.0
libcni Version:
image Version: 5.38.0
Git Commit:
Built: Wed Dec 3 14:00:34 2025
OS/Arch: linux/amd64
BuildPlatform: linux/amd64buildah info output
buildah info
{
"host": {
"CgroupVersion": "v2",
"Distribution": {
"distribution": "arch",
"version": "unknown"
},
"MemFree": 2375786496,
"MemTotal": 33286086656,
"OCIRuntime": "crun",
"SwapFree": 8557125632,
"SwapTotal": 8589930496,
"arch": "amd64",
"cpus": 16,
"hostname": "marmis-arch",
"kernel": "6.18.2-zen2-1-zen",
"os": "linux",
"rootless": true,
"uptime": "28h 37m 9.75s (Approximately 1.17 days)",
"variant": ""
},
"store": {
"ContainerStore": {
"number": 0
},
"GraphDriverName": "overlay",
"GraphImageStore": "",
"GraphOptions": null,
"GraphRoot": "/home/marmis/.local/share/containers/storage",
"GraphStatus": {
"Backing Filesystem": "extfs",
"Native Overlay Diff": "true",
"Supports d_type": "true",
"Supports shifting": "false",
"Supports volatile": "true",
"Using metacopy": "false"
},
"GraphTransientStore": false,
"ImageStore": {
"number": 57
},
"RunRoot": "/run/user/1000/containers"
}
}podman run --privileged --rm -i quay.io/buildah/stable:latest buildah info
{
"host": {
"CgroupVersion": "v2",
"Distribution": {
"distribution": "fedora",
"version": "43"
},
"MemFree": 2334232576,
"MemTotal": 33286086656,
"OCIRuntime": "crun",
"SwapFree": 8557125632,
"SwapTotal": 8589930496,
"arch": "amd64",
"cpus": 16,
"hostname": "afcdbb9a9ffb",
"kernel": "6.18.2-zen2-1-zen",
"os": "linux",
"rootless": true,
"uptime": "28h 35m 34.76s (Approximately 1.17 days)",
"variant": ""
},
"store": {
"ContainerStore": {
"number": 0
},
"GraphDriverName": "overlay",
"GraphImageStore": "",
"GraphOptions": [
"overlay.imagestore=/var/lib/shared",
"overlay.imagestore=/usr/lib/containers/storage",
"overlay.mount_program=/usr/bin/fuse-overlayfs",
"overlay.mountopt=nodev,fsync=0"
],
"GraphRoot": "/var/lib/containers/storage",
"GraphStatus": {
"Backing Filesystem": "extfs",
"Native Overlay Diff": "false",
"Supports d_type": "true",
"Supports shifting": "true",
"Supports volatile": "true",
"Using metacopy": "false"
},
"GraphTransientStore": false,
"ImageStore": {
"number": 0
},
"RunRoot": "/run/containers/storage"
}
}Provide your storage.conf
Default file from Arch Linux:
/etc/containers/storage.conf
# This file is the configuration file for all tools
# that use the containers/storage library. The storage.conf file
# overrides all other storage.conf files. Container engines using the
# container/storage library do not inherit fields from other storage.conf
# files.
#
# Note: The storage.conf file overrides other storage.conf files based on this precedence:
# /usr/containers/storage.conf
# /etc/containers/storage.conf
# $HOME/.config/containers/storage.conf
# $XDG_CONFIG_HOME/containers/storage.conf (if XDG_CONFIG_HOME is set)
# See man 5 containers-storage.conf for more information
# The "storage" table contains all of the server options.
[storage]
# Default storage driver, must be set for proper operation.
driver = "overlay"
# Temporary storage location
runroot = "/run/containers/storage"
# Priority list for the storage drivers that will be tested one
# after the other to pick the storage driver if it is not defined.
# driver_priority = ["overlay", "btrfs"]
# Primary Read/Write location of container storage
# When changing the graphroot location on an SELinux system, you must
# ensure the labeling matches the default location's labels with the
# following commands:
# semanage fcontext -a -e /var/lib/containers/storage /NEWSTORAGEPATH
# restorecon -R -v /NEWSTORAGEPATH
graphroot = "/var/lib/containers/storage"
# Optional alternate location of image store if a location separate from the
# container store is required. If set, it must be different than graphroot.
# imagestore = ""
# Storage path for rootless users
#
# rootless_storage_path = "$HOME/.local/share/containers/storage"
# Transient store mode makes all container metadata be saved in temporary storage
# (i.e. runroot above). This is faster, but doesn't persist across reboots.
# Additional garbage collection must also be performed at boot-time, so this
# option should remain disabled in most configurations.
# transient_store = true
[storage.options]
# Storage options to be passed to underlying storage drivers
# AdditionalImageStores is used to pass paths to additional Read/Only image stores
# Must be comma separated list.
additionalimagestores = [
]
# Options controlling how storage is populated when pulling images.
[storage.options.pull_options]
# Enable the "zstd:chunked" feature, which allows partial pulls, reusing
# content that already exists on the system. This is disabled by default,
# and must be explicitly enabled to be used. For more on zstd:chunked, see
# https://github.com/containers/storage/blob/main/docs/containers-storage-zstd-chunked.md
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
# enable_partial_images = "false"
# Tells containers/storage to use hard links rather then create new files in
# the image, if an identical file already existed in storage.
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
# use_hard_links = "false"
# Path to an ostree repository that might have
# previously pulled content which can be used when attempting to avoid
# pulling content from the container registry.
# ostree_repos=""
# If set to "true", containers/storage will convert images that are
# not already in zstd:chunked format to that format before processing
# in order to take advantage of local deduplication and hard linking.
# It is an expensive operation so it is not enabled by default.
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
# convert_images = "false"
# This should ALMOST NEVER be set.
# It allows partial pulls of images without guaranteeing that "partial
# pulls" and non-partial pulls both result in consistent image contents.
# This allows pulling estargz images and early versions of zstd:chunked images;
# otherwise, these layers always use the traditional non-partial pull path.
#
# This option should be enabled EXTREMELY rarely, only if ALL images that could
# EVER be conceivably pulled on this system are GUARANTEED (e.g. using a signature policy)
# to come from a build system trusted to never attack image integrity.
#
# If this consistency enforcement were disabled, malicious images could be built
# in a way designed to evade other audit mechanisms, so presence of most other audit
# mechanisms is not a replacement for the above-mentioned need for all images to come
# from a trusted build system.
#
# As a side effect, enabling this option will also make image IDs unpredictable
# (usually not equal to the traditional value matching the config digest).
# insecure_allow_unpredictable_image_contents = "false"
# Root-auto-userns-user is a user name which can be used to look up one or more UID/GID
# ranges in the /etc/subuid and /etc/subgid file. These ranges will be partitioned
# to containers configured to create automatically a user namespace. Containers
# configured to automatically create a user namespace can still overlap with containers
# having an explicit mapping set.
# This setting is ignored when running as rootless.
# root-auto-userns-user = "storage"
#
# Auto-userns-min-size is the minimum size for a user namespace created automatically.
# auto-userns-min-size=1024
#
# Auto-userns-max-size is the maximum size for a user namespace created automatically.
# auto-userns-max-size=65536
[storage.options.overlay]
# ignore_chown_errors can be set to allow a non privileged user running with
# a single UID within a user namespace to run containers. The user can pull
# and use any image even those with multiple uids. Note multiple UIDs will be
# squashed down to the default uid in the container. These images will have no
# separation between the users in the container. Only supported for the overlay
# and vfs drivers.
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
#ignore_chown_errors = "false"
# Inodes is used to set a maximum inodes of the container image.
# inodes = ""
# Path to an helper program to use for mounting the file system instead of mounting it
# directly.
#mount_program = "/usr/bin/fuse-overlayfs"
# mountopt specifies comma separated list of extra mount options
mountopt = "nodev"
# Set to skip a PRIVATE bind mount on the storage home directory.
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
# skip_mount_home = "false"
# Set to use composefs to mount data layers with overlay.
# This is a "string bool": "false" | "true" (cannot be native TOML boolean)
# use_composefs = "false"
# Size is used to set a maximum size of the container image.
# size = ""
# ForceMask specifies the permissions mask that is used for new files and
# directories.
#
# The values "shared" and "private" are accepted.
# Octal permission masks are also accepted.
#
# "": No value specified.
# All files/directories, get set with the permissions identified within the
# image.
# "private": it is equivalent to 0700.
# All files/directories get set with 0700 permissions. The owner has rwx
# access to the files. No other users on the system can access the files.
# This setting could be used with networked based homedirs.
# "shared": it is equivalent to 0755.
# The owner has rwx access to the files and everyone else can read, access
# and execute them. This setting is useful for sharing containers storage
# with other users. For instance have a storage owned by root but shared
# to rootless users as an additional store.
# NOTE: All files within the image are made readable and executable by any
# user on the system. Even /etc/shadow within your image is now readable by
# any user.
#
# OCTAL: Users can experiment with other OCTAL Permissions.
#
# Note: The force_mask Flag is an experimental feature, it could change in the
# future. When "force_mask" is set the original permission mask is stored in
# the "user.containers.override_stat" xattr and the "mount_program" option must
# be specified. Mount programs like "/usr/bin/fuse-overlayfs" present the
# extended attribute permissions to processes within containers rather than the
# "force_mask" permissions.
#
# force_mask = ""Upstream Latest Release
Yes
Additional environment details
Running from Arch Linux and with the official Buildah image.
Additional information
Always reproducible with the specified .dockerignore file.