ca-derivations/linux: built kernel not bootable

[j-baker]

Nixpkgs version

• Stable (24.11)

Describe the bug

I'm finding that when I turn on ca-derivations for my whole system, the resulting kernel image isn't bootable. I get a message saying 'Failed to decompress kernel' on boot. Digging in, this comes from the efi stub, and is basically the routine that decompresses the kernel before executing it. I find that I can still decompress the offending kernels and initrds using the 'decompress-vmlinux' from linux-scripts, so it's not immediately clear what's wrong, but I'm basically wondering if the part of the ca derivations where the output gets modified is breaking the integrity of some part of the linux kernel decompression routine.

Steps to reproduce

enable ca derivations for nixos system, it will no longer be bootable

Expected behaviour

still boots

Screenshots

No response

Relevant log output

Additional context

No response

System metadata

• system: "x86_64-linux"
• host os: Linux 6.12.8, NixOS, 24.11 (Vicuna), 24.11.20250106.3f0a8ac
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.24.11
• nixpkgs: /nix/store/fz4h8yz3qr83p6cfhisgj02knjqg6nxs-source

Notify maintainers

---

Note for maintainers: Please tag this issue in your pull request description. (i.e. Resolves #ISSUE.)

I presume this is really for the ca-derivations maintainers.

I assert that this issue is relevant for Nixpkgs

• I assert that this is a bug and not a support request.
• I assert that this is not a duplicate of an existing issue.
• I assert that I have read the NixOS Code of Conduct and agree to abide by it.

Is this issue important to you?

Add a 👍 reaction to issues you find important.

[j-baker]

Update: I dug in a bit more and modified 'extract-vmlinux' like so:

try_decompress()
{
	# The obscure use of the "tr" filter is to work around older versions of
	# "grep" that report the byte offset of the line instead of the pattern.

	# Try to find the header ($1) and decompress from here
	for	pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
	do
		pos=${pos%%:*}
	        tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
		>&2 echo $tmp $3
		tail -c+$pos "$img" | $3 -t # this bit is new
		check_vmlinux $tmp
	done
}

here '$3' is the decompressor e.g. unzstd, so the -t generally means 'run with integrity checks'. My assumption was that linux might do integrity checks on the kernel decompress because that would be sane.

We would expect a bunch of errors as the way the script works is to try decompressing with each algorithm. When decompressing the 'before' and 'after' kernel files, I see, on one:

./extract-vmlinux /boot/EFI/nixos/kernel-6.12.8-qoiuszlcwe2g33vo2yru65obglimoxq44izwm4mi5fufx4mvrwvq.efi > /dev/null
/tmp/vmlinux-8O4 gunzip
gzip: stdin is encrypted -- not supported
/tmp/vmlinux-8O4 gunzip
gzip: stdin: header checksum 0xb2b8 != computed checksum 0x27b0
/tmp/vmlinux-8O4 gunzip
gzip: stdin has flags 0x9b -- not supported
/tmp/vmlinux-8O4 lzop -d
lzop: <stdin>: not a lzop file
/tmp/vmlinux-8O4 unzstd
/*stdin*\            : 0 B...     zstd: /*stdin*\: unsupported format

and on the ca-derivations version

./extract-vmlinux /boot/EFI/nixos/kernel-6.12.8-hwfhfbh6ff3lo3qzsgw675vaone4ugl5ykuagibjbgoop7zezmea.efi > /dev/null
/tmp/vmlinux-ERB gunzip
gzip: stdin is encrypted -- not supported
/tmp/vmlinux-ERB gunzip
gzip: stdin: header checksum 0xb2b8 != computed checksum 0x27b0
/tmp/vmlinux-ERB gunzip
gzip: stdin has flags 0x9b -- not supported
/tmp/vmlinux-ERB lzop -d
lzop: <stdin>: not a lzop file
/tmp/vmlinux-ERB unzstd
/*stdin*\            : 0 B...     /*stdin*\ : Decoding error (36) : Restored data doesn't match checksum

so I can only assume that the fixup work in ca-derivations is causing a zstd checksum to fail, and this is causing linux to barf on startup.

Either way, even though my system actually builds, it doesn't boot :(.

[JohnRTitor]

CC @K900