Debian 12 "bookworm"

[darkbasic]

@rlaager is it safe to follow the Debian 11 Bullseye Root on ZFS HOWTO for Debian 12? Is there any change planned for Debian 12 "bookworm"?

[rlaager]

It's probably safe. I haven't had a chance to test it. I've been too busy with other things in life. Sorry!

I do intend on testing it soon and cutting a new version of the guide.

[jim-perkins]

Thanks for the very good tutorial. Just went thru it on bookworm. Here are my notes. If you would like pull requests just let me know and if you would like it all in one request or seperate request for each change. thanks again

Step 1: Prepare The Install Environment

1. Boot the Debian GNU/Linux Live CD.
   debian-live-12.0.0-amd64-standard.iso
   note- this is the live install image used

I actually did step 3 here, as my ipmi interface isnt cut and paste

2. Setup and update the repositories:
   deb http://deb.debian.org/debian/ bookworm main non-free-firmware contrib
   deb-src http://deb.debian.org/debian/ bookworm main non-free-firmware contrib
   deb [trusted=yes] file:/run/live/medium bookworm main non-free-firmware
   note- I had read in the release notes about the new non-free-firmware section

4 . Disable automounting:
-bash: gsettings: command not found
note- probably related to the image I used in step 1

Step 2: Disk Formatting
4. Create the boot pool:
${DISK}-part3
note – when using a kvm virt machine and /dev/vdx the partitions are /dev/vdxy
although this isnt related to bullseye vs bookworm

Step 3: System Installation

5. Install the minimal system:
   debootstrap bookworm /mnt
   note- updated to bookworm

Step 4: System Configuration

3. Configure the package sources:

deb http://deb.debian.org/debian bookworm main contrib non-free-firmware
deb-src http://deb.debian.org/debian bookworm main contrib non-free-firmware

deb http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware
deb-src http://deb.debian.org/debian-security bookworm-security main contrib non-free-firmware

deb http://deb.debian.org/debian bookworm-updates main contrib non-free-firmware
deb-src http://deb.debian.org/debian bookworm-updates main contrib non-free-firmware

note- updated from bullseye to bookworm and added non-free-firmware

5. Configure a basic system environment:
   ln -s /proc/self/mounts /etc/mtab
   ln: failed to create symbolic link '/etc/mtab': File exists
   note – got this error but things seemed to work fine

6. Install an NTP service to synchronize time
   timedatectl
   System has not been booted with systemd as init system (PID 1). Can't operate.
   Failed to connect to bus: Host is down
   note-got this error but things seemed to work fine

15: Optional: For ZFS native encryption or LUKS, configure Dropbear for remote unlocking:
the location for configuration of dropbear has moved to
Note: Earlier versions of dropbear-initramfs stored configuration files under /etc/dropbear-initramfs. In the version I'm using (v2020.81) this has moved to /etc/dropbear/initramfs and is what I'll be referencing. Substitute the path to suit your installed version.

[rlaager]

Awesome! If you are willing to help with this effort, I'll certainly take that help.

First off, see this:
https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Bullseye%20Root%20on%20ZFS.html#contributing

The changes will always include:

1. Copy the Bullseye file to a version for Bookworm.
2. Link it from the index page.
3. Add the "Newer release available" section to Bullseye. (See Buster for an example.) This will link to the Bookworm instructions.
4. Change all instances of Bullseye to Bookworm, e.g. in sources files.
5. Update the ISO link.

Then it comes down to whatever actually changed. You listed some:

• gsettings... That's really bizarre. gsettings is in libglib2.0-bin. That really doesn't exist in the live environment? Is it a GNOME environment? If it's a GNOME environment and gsettings really is not installed, then I suppose we need to add a sudo apt install libglib2.0-bin before that line. (I can't imagine automounting is disabled by default in the live environment, so we still need to be able to change that setting.)
• /etc/mtab... If it exists by default, we can just drop that step.
• timedatectl... does it work outside of the chroot (i.e. if you just boot a live environment, does it work)?
• dropbear... if the path has changed, definitely update it.

One PR is fine.

Even one commit is fine for something like this. If you really wanted to do multiple commits, I think the logical way to split them up would be the one that copies the instructions and makes the "always include" changes from above and subsequent commits for the actual changes.

[jim-perkins]

Awesome! If you are willing to help with this effort, I'll certainly take that help.

Work in progress here. I will make a pull request when it is complete
https://github.com/jim-perkins/openzfs-docs/tree/Debian-Bookworm

I am a baby using git so the commit history may be funky but it will all be good in the end. Probably finish in the next day or 2.

The changes will always include:

2. Link it from the index page.

Happened automatically

5. Update the ISO link.

It already linked to 'current' so we're good here
https://cdimage.debian.org/mirror/cdimage/release/current-live/amd64/iso-hybrid/

Then it comes down to whatever actually changed. You listed some:

* gsettings... That's really bizarre. gsettings is in libglib2.0-bin. That really doesn't exist in the live environment? Is it a GNOME environment? If it's a GNOME environment and gsettings really is not installed, then I suppose we need to add a `sudo apt install libglib2.0-bin` before that line. (I can't imagine automounting is disabled by default in the live environment, so we still need to be able to change that setting.)

Yea I dfdnt read the instructions well and downloaded the 'standard' iso which doesnt include X11/GUI . the 'Gnome' version has gsettings. We can let this go . The tutorial doesnt need Xll though, worked fine for me from the command line.

* /etc/mtab... If it exists by default, we can just drop that step.

yes it existed in the chroot. so I will drop it

* timedatectl... does it work outside of the chroot (i.e. if you just boot a live environment, does it work)?

yes it was only in the chroot that it gave an error. Not sure how to handle this so will let it be for now

* dropbear... if the path has changed, definitely update it.

Yes the path has changed . This one took me a bit to figure out.

-- a question --
What about the non-free-firmware component ? I can add it to the sources.list or leave it out. I added it to all and didnt get any errors.
https://wiki.debian.org/Firmware#Debian_12_.28bookworm.29_and_later

[darkbasic]

#433
Isn't this something we should also take into consideration for bookworm?

[rlaager]

Don't worry about the git history. If you end up with a million fix commits, that's fine. I can squash together for the merge.

I generally use the GUI version of the live environment. I figure that's more user friendly. Even in the case of people that "know what they are doing", it can be nice (e.g. having a web browser to look at the instructions and copy-and-paste).

We should include non-free-firmware, as Debian does that by default. People that don't want it can remove it.

Yes, we should pull in the changes from PR #433.

[jim-perkins]

Made a PR. Never tested a EUFI installation. Tested a kvm virt install and a physical install , both with 2 disks in a mirror situation, Had to -f the zfs creates on reused zfs disks, even after wiping per the instructions. but maybe that was just me. DIdnt put that in the changes.

It seems we are aware that the zfs rpool fails to export coming out of the chroot and the first install always requires a zpool import -f rpool . Never tested if that was possible via the dropbear initramfs ssh .

Let me know if any follow up is needed. thanks again for all the work with open zfs

[rlaager]

It seems we are aware that the zfs rpool fails to export coming out of the chroot and the first install always requires a zpool import -f rpool

Indeed. My plan (which I've just not had time to get to) is to add notes on this to the instructions. I just can't find any way to get it to unmount cleanly!

[darkbasic]

What's your experience with zfsbootmenu? I need the ability to boot into old snapshots and eventually rollback so it looks like a good alternative to grub. I used to use zsys in the past but it's an unmaintained nightmare nowadays.

[rlaager]

I haven't personally used it. I believe Jim (author of syncoid, etc.) recommends it. My "problem" with it is that getting it into the distro would be a pain, because to my understanding, it's basically another build of Linux.

That said, while I try to follow stock distro practices, it'd probably be fine to have this an optional add-on.

[darkbasic]

My "problem" with it is that getting it into the distro would be a pain, because to my understanding, it's basically another build of Linux.

There are binaries available, it's basically a curl command away.

[colmbuckley]

Step 2: Disk Formatting 4. Create the boot pool: ${DISK}-part3 note – when using a kvm virt machine and /dev/vdx the partitions are /dev/vdxy although this isnt related to bullseye vs bookworm

Note that the creation of the bpool pool can be simplified using -o compatibility=grub2 rather than -d and the long list of features - ie:

root@debian:~# zpool create \
    -o ashift=12 \
    -o autotrim=on \
    -o cachefile=/etc/zfs/zpool.cache \
    -o compatibility=grub2 \
    -O devices=off \
    -O acltype=posixacl -O xattr=sa \
    -O compression=lz4 \
    -O normalization=formD \
    -O relatime=on \
    -O canmount=off -O mountpoint=/boot -R /mnt \
    bpool ${DISK}3

HTH.

[FingerlessGlov3s]

I recently followed the Bullseye guide for installing bookworm. I didn't have any major problems. I made note of all the commands I ran.

The two things that I did have to do but I don't believe the guide mentions it. After doing the first boot. After doing all the chroot bits. Both the bpool and rpool fail to import because the hostid has changed. So when it drops in to emergency mode on the initramfs. Just do a zpool import -f {poolname} for each pool, then type exit. Then the install works as intended and any reboots after this point boot up as expected.

I have got the bpool talking about doing a pool upgrade, not sure if that happens in Bullseye, but could be a Bookworm thing, I wish I tried the updated bpool creation command @colmbuckley mentioned, but the server I did the install one has hit production now, so can't do a reinstall to test 😅

Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.

[colmbuckley]

I wish I tried the updated bpool creation command @colmbuckley mentioned, but the server I did the install one has hit production now, so can't do a reinstall to test 😅

You can still do zpool set compatibility=grub2 bpool on this system, even after installation and boot; that is a very low-risk operation. That should address the messages about upgrading; but it does not itself enable or disable any features, so the warning you quote should not be an issue.

Hope this helps.

[FingerlessGlov3s]

I've found some time to recreate my environment in a VM, and after doing the command you mentioned above, the upgrade warning changes the below.

Inside /usr/share/zfs/compatibility.d/ directory grub2 is listed.
Do I need to do something else to mirror the features from the grub2 files exactly?

  pool: bpool
 state: ONLINE
status: One or more features are enabled on the pool despite not being
        requested by the 'compatibility' property.
action: Consider setting 'compatibility' to an appropriate value, or
        adding needed features to the relevant file in
        /etc/zfs/compatibility.d or /usr/share/zfs/compatibility.d.
config:

        NAME                                            STATE     READ WRITE CKSUM
        bpool                                           ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-part3  ONLINE       0     0     0
            scsi-0QEMU_QEMU_HARDDISK_drive-scsi1-part3  ONLINE       0     0     0

[colmbuckley]

Yes; the instructions added two new features to the bpool pool which are not yet reflected in the grub2 compatibility file - there is a PR for this merged upstream but it hasn't yet been included in a release.

To eliminate this warning (harmlessly), you can do:

# mkdir -p /etc/zfs/compatibility.d
# cp /usr/share/zfs/compatibility.d/grub2 /etc/zfs/compatibility.d
# (echo livelist; echo zpool_checkpoint) >> /etc/zfs/compatibility.d/grub2
# zpool status bpool

[FingerlessGlov3s]

I can confirm doing what you said has fixed the status messages,

When you mean upstream do you mean Debian or openzfs itself?

[colmbuckley]

I mean OpenZFS - this PR. That adds these two features to the system grub2 compatibility file, so there's no need to create a local one (in /etc) as above.

[jim-perkins]

My pull request was merged and there is now Debian 12 Bookworm Root on ZFS documentation. thanks so much. Should this issue be closed and marked as resolved ?