build failure with podman

[jasonbrooks]

During the coreos-assembler build process on a Fedora 29 VM with nested kvm enabled, I'm getting a ton of cp: setting attributes for XXX: Operation not supported errors before the operation errors out. This is with podman, but with docker the operation goes further (It's stopping much later on, with a ModuleNotFoundError: No module named 'pytz' error -- I'll follow up on that separately).

dracut: FAILED: /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs /usr/lib/dracut/modules.d/30ignition/ignition-files.service /usr/lib/systemd/system/ignition-files.service
cp: setting attributes for '/tmp/dracut/dracut.HD3Ged/initramfs/usr/lib/systemd/system/ignition-ask-var-mount.service': Operation not supported
dracut-install: ERROR: installing '/usr/lib/dracut/modules.d/30ignition/ignition-ask-var-mount.service' to '/usr/lib/systemd/system/ignition-ask-var-mount.service'
dracut: FAILED: /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs /usr/lib/dracut/modules.d/30ignition/ignition-ask-var-mount.service /usr/lib/systemd/system/ignition-ask-var-mount.service
cp: setting attributes for '/tmp/dracut/dracut.HD3Ged/initramfs/lib/modules/4.19.8-300.fc29.x86_64//kernel/drivers/firmware/qemu_fw_cfg.ko.xz': Operation not supported
dracut-install: dracut_install 'qemu_fw_cfg' ERROR
dracut-install: ERROR: installing 'qemu_fw_cfg'
dracut: FAILED:  /usr/lib/dracut/dracut-install -D /tmp/dracut/dracut.HD3Ged/initramfs --kerneldir /lib/modules/4.19.8-300.fc29.x86_64/ -m qemu_fw_cfg
dracut: installkernel failed in module ignition
error: Finalizing rootfs: During kernel processing: Executing bwrap(rpmostree-dracut-wrapper): Child process killed by signal 1

[jlebon]

ModuleNotFoundError: No module named 'pytz'

Ahh yeah, that one is likely a regression from #252.

[cgwalters]

cp: setting attributes for '/tmp/dracut/dracut.HD3Ged/initramfs/usr/lib/systemd/system/ignition-ask-var-mount.service': Operation not supported

Is currently expected...it's a dracut bug that should be harmless.

[jlebon]

#255

[dustymabe]

#255

merged.

[dustymabe]

error: Finalizing rootfs: During kernel processing: Executing bwrap(rpmostree-dracut-wrapper): Child process killed by signal 1

can you maybe attach a file to this ticket that is your full log output including the invocation using podman/docker ?

[jasonbrooks]

@dustymabe Yeah, I'm re-running it

https://paste.fedoraproject.org/paste/kSK6ErnDUkdTUYF9OP2DEw

[jlebon]

cp: setting attributes for '/tmp/dracut/dracut.D5H9tj/initramfs/lib/modules/4.19.8-300.fc29.x86_64//kernel/drivers/firmware/qemu_fw_cfg.ko.xz': Operation not supported
dracut-install: dracut_install 'qemu_fw_cfg' ERROR
dracut-install: ERROR: installing 'qemu_fw_cfg'

Hmm, yeah this looks like dracut's cp failing to copy xattrs.

This is with podman, but with docker the operation goes further

Can you paste the outputs of findmnt inside the container for the two? Looks like the difference might be podman setting a tmpfs on /tmp?

Alternatively, you might also want to try the new rootless backend, see #236 (comment).

[cgwalters]

I think the problem here has to do with /var/tmp. I can't reproduce this in my devshell but I can with the official container + alias.

I think rpm-ostree should always use a tmpdir under the target root; but looking at this now.

[cgwalters]

PR in coreos/rpm-ostree#1716

[dustymabe]

PR in projectatomic/rpm-ostree#1716

there is a new build of coreos-assembler I kicked off this morning that should have this fix if anyone wants to try things again.

[miabbott]

I just ran into this after updating my Silverblue host:

podman 1:0.10.1.3-4.gitdb08685.fc29 -> 1:0.12.1.1-1.git66d3499.fc29

But the latest quay.io/coreos-assembler/coreos-assembler build that @dustymabe did fixed it for me 👍

[jasonbrooks]

This is working for me now -- I'm most of the way through a build w/ podman. I pulled the latest image this afternoon.

[jlebon]

Cool, closing this issue then!