tools: labs: qemu: Add run-qemu.sh as alternative

TL;DR: In one window run `make -j$(ncproc) console` and `cd` to some
lab's subdirectory (`skels/...`). In second window `cd` to matching
`skels/...` subdirectory, edit source files and compile with something
like `kmake` (`alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'`) as
needed. The `skels` directory is shared between the host and the guest,
thus in the first window, so you can just `insmod` and `rmmod` the
compiled modules. You can kill the VM by `CTRL-a x` (if you made some
writes from the VM it might be a good idea to run `sync` first). Samba
and QEMU are required.

Full description:

To be honest I don't like the current QEMU setup. I am sure there are
things it does that I don't understand yet, because I have not yet
finished all the labs, but in any case I think that the setup can be
improved.

Some things in particular I don't like about the current setup:

 - "Huge" opaque `.ext4` images are used, even though the contents of
   the  root file system are not that large.
 - While running QEMU newly built modules can't be copied to the image.
 - Mounting and unmounting the `.ext4` image for copying the modules
   requires `sudo`.
 - The networking setup seems too complex, requires `sudo` and was
   broken (at least for me - IIRC I didn't get IP through DHCP), thus I
   also didn't get `ssh` to work. I also seem to be not the only one
   having issues with this:
   linux-kernel-labs#240 (comment)
 - `dnsmasq` and `nttctp` mostly don't start correctly (they are not
   killed correctly by the previous run) - this isn't a problem on my
   end, as demonstrated by the output at
   https://linux-kernel-labs.github.io/refs/heads/master/info/vm.html,
   which shows the same issues.
 - Running `minicom` is required to access the serial port, thus at
   least two terminals are required for working with the VM (not a huge
   problem for me personally, since I use `tmux`, but I know some people
   complain about this). The setup also seems unnecessarily complex.
 - I remember a lot of the `.mon` `.pts` files being left uncleaned in
   some cases.
 - I recall warnigns about some of the entries added to `/etc/inittab`
   being ignored.
 - Even though root login requires no password I have to enter the
   `root` username.

In this commit I introdoce an alternative way of running QEMU through a
new `run-qemu.sh` script. The setup is laregely independent and its user
interface consists of `console` and `gui` targets. I tried to make the
script parameterizable through environment variables (inherited from
Make variables), though it may be argued that the default values should
be encoded in Makefile rather than in the script like they are now. I
have no strong opinions about that, it's' just that the current state
allows running the script in standalone fashion.

What the setup brings:

 - A rootfs is extracted from the official Yocto Project tarball and
   kept in a directory that is shared through  [Samba as network
   share](https://www.kernel.org/doc/html/latest/filesystems/cifs/cifsroot.html).
   The `skels` directory is shared as well. Thus the modules can be
   freely tweaked / compiled / ran from either the host or guest.
 - The QEMU stdio serial console setup is used (`ttyS0` on the kernel
   side). This means that running QEMU results in the serial console
   being mapped directly to standard input and output of the terminal -
   `minicom` is not needed. This is the console mode (`make console`).
 -  The setup allows also allows the virtual machine to be run in
    graphical mode (`make gui`).
 - Root is logged in automatically in `console` mode (though similar
   thing could be done for the `gui` mode).
 - Although Samba (`smbd`) is required, `sudo` or root access is not.
 - Networking through QEMU's default [SLIRP backend](https://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29).
   DHCP is handled by the kernel, which overcomes some problems I had
   with the System V init system in the Yocto images.
 - The compilation can largely be done with something like this `kmake`
   alias: `alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'`
   (customize as needed). Though this is not enough for some labs (I no
   longer remember the details, but I think it was some of the earlier
   labs which had dependencies between modules, I think I used the
   classic `make build` for that.

Known issues:

 - SSH support is currently missing. This both requires more featureful
   Yocto images and is IMO unnecessary, since it wouldn't bring much
   benefit over the console mode. Though it can be easily achieved by
   using QEMU option like `-nic user,hostfwd=tcp::2222-:22`, which would
   allow SSH'ing into the guest by something like `ssh -p 2222
   root@localhost`.
 - I used a slightly less advanced setup while doing the labs, so the
   lab workflow with this particular setup is largely untested. There
   may be problems with file permissions due to the samba share.
 - The guest seems to fail to shutdown correctly in a timely manner. I
   just took the habbit of killing qemu with `CTRL-a` followed by `x`,
   potentially running `sync` first to ensure my work is saved (though
   rarely did I actually modify anything on the guest side).

[The former setup](720bd64)
I used contains some details of the SSH setup if anyone is interested in
that. It was the basis for this PR, so some ideas can be seen there
(Samba share for `skels`), but I didn't take particular care with [the
kernel config](0290919)
and the automounting didn't really work (the `init` would try to mount
the filesystem before networking was up).

What I evaluated and didn't use in the end:

 - At first I tried to extend my former setup by just automounting the
   Samba share. I didn't manage to do this - the (non)workings of init
   scripts seem to be beyond me. If anyone is interested here are a few
   pointers:
   [[1]](https://unix.stackexchange.com/questions/169697/how-does-netdev-mount-option-in-etc-fstab-work),
   `/etc/inittab`, `/etc/init.d/mountall.sh`, `/etc/init.d/mountnfs.sh`.
 - I tried using `9p` [[2]](https://wiki.qemu.org/Documentation/9p),
   [[3]](https://wiki.qemu.org/Documentation/9psetup)
   [[4]](https://wiki.qemu.org/Documentation/9p_root_fs) which is built
   into QEMU and can be compiled into the kernel. With `mapped-xattr`
   security model it would be too cumbersome to create the rootfs, and
   `passthrough` would require root privileges. It is also very slow.
   There are also some problems with trying to use it as rootfs, maybe
   specific to `linux-kernel-labs` kernel version or config. Ask me if
   interested.
   [[5]](https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg07184.html)
   [[6]](https://lore.kernel.org/linux-fsdevel/[email protected]/)
   [[7]](https://lore.kernel.org/all/[email protected]/T/)
 - QEMU has an option to setup the Samba share on its own, though I
   found a custom config (based on the QEMU one) to be easier - allows
   customization like multiple shares, unix extensions, different port,
   etc.

Author: vlasakm

tools/labs/.gitignore

@@ -6,5 +6,7 @@ rootfs.img
 disk1.img
 disk2.img
 /*core-image-*.ext4
+/*core-image-*.bz2
 .modinst
 /out/
+/rootfs

tools/labs/qemu/Makefile

@@ -95,4 +95,41 @@ clean::
 	-rm -f disk1.img disk2.img
 	-rm -f pipe1.in pipe1.out pipe2.in pipe2.out
 
-.PHONY: boot gdb clean tap0 tap1
+# Run with one of the following:
+#     make -j$(nproc) console
+#     make -j$(nproc) gui
+
+# Attach debugger with
+#     make gdb
+
+# Stop with
+#     sync # make sure all filesystem changes are propagated
+#     CTRL-A X # kill qemu, faster and more reliable than poweroff etc.
+
+# Compile in skel directories with something like:
+#     alias kmake='make -C "$HOME/src/linux/" M="$(pwd)"'
+#     kmake
+
+YOCTO_ROOTFS = core-image-minimal-qemu$(ARCH).tar.bz2
+
+console: $(ZIMAGE) rootfs
+	MODE=console qemu/run-qemu.sh
+
+gui:
+	MODE=gui qemu/run-qemu.sh
+
+rootfs: $(YOCTO_ROOTFS)
+	mkdir -p rootfs
+	tar -xf core-image-minimal-qemux86.tar.bz2 -C rootfs
+	sed -i 's@/sbin/getty@& -n -l "/sbin/rootlogin"@' rootfs/bin/start_getty
+	printf '%s\n' '#!/bin/sh' '/bin/login -f root' > rootfs/sbin/rootlogin
+	chmod +x rootfs/sbin/rootlogin
+	mkdir -p rootfs/home/root/skels
+	echo "//10.0.2.2/skels /home/root/skels cifs port=4450,guest,user=dummy 0 0" >> rootfs/etc/fstab
+
+$(YOCTO_ROOTFS):
+	wget $(YOCTO_URL)/$(YOCTO_ROOTFS)
+
+
+
+.PHONY: console gui boot gdb clean tap0 tap1

tools/labs/qemu/kernel_config.x86

@@ -97,3 +97,18 @@ CONFIG_UPROBE_EVENTS=y
 CONFIG_LOCKDEP=y
 # kernel lock tracing:
 CONFIG_LOCK_STAT=y
+
+# for qemu serial console ttyS0:
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+
+# for qemu default netdev:
+CONFIG_E1000=y
+
+# for CIFS/SMB/samba rootfs and share
+CONFIG_IP_PNP=y
+CONFIG_IP_PNP_DHCP=y
+CONFIG_CIFS=y
+CONFIG_CIFS_XATTR=y
+CONFIG_CIFS_POSIX=y
+CONFIG_CIFS_ROOT=y

tools/labs/qemu/run-qemu.sh

@@ -0,0 +1,114 @@
+#!/bin/bash
+
+die() { echo "$0: error: $@" >&2; exit 1; }
+
+script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null && pwd)"
+
+base_dir=${ROOT:-"$(readlink -f "$script_dir/..")"}
+arch=${ARCH:-"x86"}
+kernel=${ZIMAGE:-"$(readlink -f "$base_dir/../../arch/$arch/boot/bzImage")"}
+rootfs=${ROOTFS:-"$(readlink -f "$base_dir/rootfs")"}
+skels=${SKELS:-"$(readlink -f "$base_dir/skels")"}
+
+mode="${MODE:-console}"
+case "$mode" in
+    console)
+	    qemu_display="-nographic"
+	    linux_console="console=ttyS0"
+	    ;;
+    gui)
+	    # QEMU_DISPLAY = sdl, gtk, ...
+	    qemu_display="-display ${QEMU_DISPLAY:-"sdl"}"
+	    linux_console=""
+	    ;;
+    *) echo "unknown mode '$MODE'" >&2; exit 1 ;;
+esac
+
+case "$arch" in
+    x86) qemu_arch=i386 ;;
+    arm) qemu_arch=arm ;;
+    *) echo "unknown architecture '$arch'" >&2; exit 1 ;;
+esac
+
+smbd=${SMBD:-"smbd"}
+
+qemu=${QEMU:-"qemu-system-$qemu_arch"}
+qemu_kvm=${QEMU_KVM:-"-enable-kvm -cpu host"}
+qemu_cpus=${QEMU_CPUS:-"1"}
+qemu_mem=${QEMU_MEM:-"512"}
+qemu_display=${QEMU_DISPLAY:-"$qemu_display"}
+qemu_addopts=${QEMU_ADD_OPTS:-""}
+linux_console=${LINUX_CONSOLE:-"$linux_console"}
+linux_loglevel=${LINUX_LOGLEVEL:-"15"}
+linux_term=${LINUX_TERM:-"TERM=xterm"}
+linux_addcmdline=${LINUX_ADD_CMDLINE:-""}
+
+linux_cmdline=${LINUX_CMDLINE:-"root=/dev/cifs rw ip=dhcp cifsroot=//10.0.2.2/rootfs,port=4450,guest,user=dummy $linux_console loglevel=$linux_loglevel $linux_term $linux_addcmdline"}
+
+tmp_dir=$(mktemp -d)
+user=$(id -un)
+
+cat << EOF > "$tmp_dir/smbd.conf"
+[global]
+    interfaces = 127.0.0.1
+    smb ports = 4450
+    private dir = $tmp_dir
+    bind interfaces only = yes
+    pid directory = $tmp_dir
+    lock directory = $tmp_dir
+    state directory = $tmp_dir
+    cache directory = $tmp_dir
+    ncalrpc dir = $tmp_dir/ncalrpc
+    log file = $tmp_dir/log.smbd
+    smb passwd file = $tmp_dir/smbpasswd
+    security = user
+    map to guest = Bad User
+    load printers = no
+    printing = bsd
+    disable spoolss = yes
+    usershare max shares = 0
+
+    server min protocol = NT1
+    unix extensions = yes
+
+    server role = standalone server
+    public = yes
+    writeable = yes
+    #admin users = root
+    #create mask = 0777
+    #directory mask = 0777
+    force user = $user
+    force group = $user
+
+
+[rootfs]
+    path = $rootfs
+[skels]
+    path = $skels
+EOF
+
+[ -x "$(command -v "$smbd")" ] || die "samba ('$smbd') not found"
+[ -x "$(command -v "$qemu")" ] || die "qemu ('$qemu') not found"
+
+mkdir -p "$skels"
+
+"$smbd" --no-process-group -s "$tmp_dir/smbd.conf" -l "$tmp_dir" >/dev/null 2>/dev/null &
+
+"$qemu" \
+    $qemu_kvm \
+    -smp "$qemu_cpus" -m "$qemu_mem" \
+    -no-reboot \
+    -kernel "$kernel" \
+    -append "$linux_cmdline" \
+    -gdb tcp::1234 \
+    $qemu_display \
+    $qemu_addopts
+
+# This seems to reset to the mode the terminal was prior to launching QEMU
+# Inspired by
+# https://github.com/landley/toybox/blob/990e0e7a40e4509c7987a190febe5d867f412af6/toys/other/reset.c#L26-L28
+# man 4 console_codes, ESC [ ? 7 h
+printf '\e[?7h'
+
+pkill -F "$tmp_dir/smbd.pid"
+rm -rf "$tmp_dir"