Support simplified initramfs/initramfs-less boot

[ryao]

I used to think an initramfs environment with special logic for ZFS was necessary for system boot because we could not scan /dev for devices otherwise. After answering this question again for djs in IRC on freenode, I have come to realize that is wrong.

During very early boot before the rootfs mount, we should have something like this:

kthread_run
devtmpfs_init
driver_init
kernel_init_freeable
kernel_init

That kthread_run calls devtmpfsd while devtmpfs_init waits for devtmpfsd to signal completion before proceeding:

        *err = sys_unshare(CLONE_NEWNS);
        if (*err)
                goto out;
        *err = sys_mount("devtmpfs", "/", "devtmpfs", MS_SILENT, options);
        if (*err)
                goto out;
        sys_chdir("/.."); /* will traverse into overmounted root */
        sys_chroot(".");
        complete(&setup_done);

https://github.com/torvalds/linux/blob/v4.1/drivers/base/devtmpfs.c#L376

That shows that we do have a /dev filesystem on which we can use kernel functions such as vfs_open and iterate_dir during early boot to scan the devices from inside the kernel's mount routine.

When booting ext4, we would have a stack trace like this, with ext4 using a filesystem path passed to ext4_mount to find its root device. That is the path to /dev/root, which is unfortunately hard coded in the kernel. When ext4 gets it, it is just "root" because the first 5 characters in the string are skipped:

kern_path
lookup_bdev
blkdev_get_by_path
mount_bdev
ext4_mount
mount_fs
vfs_kern_mount
do_new_mount
do_mount
sys_mount
do_mount_root
mount_block_root
mount_root
prepare_namespace
kernel_init_freeable
kernel_init

In the case of booting Linux in QEMU with an 9p-virtio rootfs without an initramfs, root=/dev/root rootfstype=9p rootflags=trans=virtio,version=9p2000.L is passed on the kernel commandline and from a cursory reading of the Linux kernel 4.1.0 source, the following stack should occur:

v9fs_mount
mount_fs
vfs_kern_mount
do_new_mount
do_mount
sys_mount
do_mount_root
mount_block_root
mount_root
prepare_namespace
kernel_init_freeable
kernel_init

We then see VFS: Mounted root (9p filesystem) readonly on device 0:17. printed to dmesg and the system boots. This only works if the 9p-virtio server in QEMU is passed id=root through an appropriate -fsdev parameter on QEMU's commandline because the first 5 characters of /dev/root are skipped.

I wrote a small patch a couple years ago that allowed an initramfs to boot the system with a similar kernel commandline and both the modules (when not kernel builtin) and zpool.cache file in the initramfs. If I recall correctly, it worked by hooking zpl_fill_super. At the time, /dev needed to be mounted and the SPL needed /sys, although /sys should no longer be a necessity. That patch was lost at some point, but it showed that we could simplify the boot process in such a way that we would not require any special treatment by the initramfs if we could scan /dev during early boot. It had been abandoned because we needed to do in-kernel device scanning and I did not think that was possible at the time.

I am filing this as a reminder for myself of these things and also to offer anyone able to implement this before I find time the opportunity. Whoever implements this will need to either handle the situation where /proc/self/mounts and also if it is not a symlink, /etc/mtab show /dev/root (or is it root?) instead of the dataset name or find a way to override it during the mount process.

I have revised this following some comments by @floppym on an earlier version of this issue.

[ryao]

We will want some way to reopen the vdevs using by-vdev, by-id or friends if the cachefile indicates that we used those previously during the boot process when userland has started.

[ngortheone]

@behlendorf why this was closed?
Is this implemented or dropped? I can't find any relevant information.

[behlendorf]

This hasn't yet been implemented. Though I agree it's an interesting idea. I've migrated it to the new GtiHub Discussions "Ideas" section. I'd like to try and start tracking feature requests there keep them separate from the other bugs.

[furbyhaxx]

any progress on this? do we still need an initrd to boot from a zfs pool?

[prrar]

That would be awesome!