[Windows Please]

[jhay06]

Description

Hi ,
This is such a great work , hoping for the Windows version

WSL is good but very slow in most cases , i prefer to have this in windows , :) hope that this will be available soon.

Thanks

[jandubois]

WSL is good but very slow in most cases

I don't see how we could improve on the performance of WSL and Hyper-V on Windows. So unless somebody else has ideas, and the ability to implement them, this is unlikely to happen.

[afbjorklund]

I think you can use the QEMU binaries with the WHPX acceleration, along with any available ISO, to see the "baseline" performance. If that is not enough, I guess this is more a request for running Windows containers (like Docker) ?

https://docs.microsoft.com/en-us/virtualization/windowscontainers/about/

I would be OK with being able to run VMs the same way on Windows, that is available on Mac and Linux today (i.e. QEMU). We had this with docker-machine and podman-machine, so it should be "possible" also for containerd-machine (lima)

But I don't run Windows myself, and when I do it is with something like MSYS

[afbjorklund]

There are some GOOS=windows compilation issues on master, but those should be easy to fix:

# github.com/lima-vm/lima/pkg/lockutil
pkg/lockutil/lockutil.go:34:27: undefined: unix.LOCK_EX
pkg/lockutil/lockutil.go:38:28: undefined: unix.LOCK_UN
pkg/lockutil/lockutil.go:48:10: undefined: unix.Flock
pkg/lockutil/lockutil.go:49:27: undefined: unix.EINTR
note: module requires Go 1.18
# github.com/lima-vm/lima/pkg/networks
pkg/networks/validate.go:76:25: undefined: syscall.Stat_t
note: module requires Go 1.18

The "lockutil" are just missing some nerdctl code available. The syscall needs wrapping...

https://github.com/containerd/nerdctl/tree/master/pkg/lockutil

---

EDIT: added:

• Add windows version of lockutil from nerdctl #911
• Gather all syscall use in the osutil helper #912

make GOOS=windows
_output/bin/limactl: PE32+ executable (console) x86-64 (stripped to external PDB), for MS Windows

[afbjorklund]

Something like: (see MSYS2, and https://www.alpinelinux.org/downloads/)

$ /c/Program\ Files/qemu/qemu-system-x86_64 -m 512 -smp 1 \
  -accel whpx,kernel-irqchip=off -cdrom alpine-virt-3.16.0-x86_64.iso
Windows Hypervisor Platform accelerator is operational

The display looks broken (no input), but -serial stdio almost works.

ISOLINUX 6.04 6.04-pre1  Copyright (C) 1994-2015 H. Peter Anvin et al
boot:


   OpenRC 0.44.10 is starting up Linux 5.15.41-0-virt (x86_64)

 * /proc is already mounted
 * Mounting /run ... * /run/openrc: creating directory
 * /run/lock: creating directory
 * /run/lock: correcting owner
 * Caching service dependencies ... [ ok ]
 * Remounting devtmpfs on /dev ... [ ok ]
 * Mounting /dev/mqueue ... [ ok ]
 * Mounting modloop  ... * Verifying modloop
 [ ok ]
 * Mounting security filesystem ... [ ok ]
 * Mounting debug filesystem ... [ ok ]
 * Mounting persistent storage (pstore) filesystem ... [ ok ]
 * Starting busybox mdev ... [ ok ]
 * Loading hardware drivers ... [ ok ]
 * Loading modules ... [ ok ]
 * Setting system clock using the hardware clock [UTC] ... [ ok ]
 * Checking local filesystems  ... [ ok ]
 * Remounting filesystems ... [ ok ]
 * Mounting local filesystems ... [ ok ]
 * Configuring kernel parameters ... [ ok ]
 * Migrating /var/lock to /run/lock ... [ ok ]
 * Creating user login records ... [ ok ]
 * Cleaning /tmp directory ... [ ok ]
 * Setting hostname ... [ ok ]
 * Starting busybox syslog ... [ ok ]
 * Starting firstboot ... [ ok ]

Welcome to Alpine Linux 3.16
Kernel 5.15.41-0-virt on an x86_64 (/dev/ttyS0)

localhost login: root
root
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

localhost:~#

---

EDIT: The kernel-irqchip thing was a workaround for a startup error:

whpx: injection failed, MSI (0, 0) delivery: 0, dest_mode: 0, trigger mode: 0, vector: 0, lost (c0350005)

And with "almost works", I mean this console has some weird issues:

localhost:~# apk add containerd
pk add containerd
-ash: pk: not found

EDIT: -display sdl works (better than "gtk")

Here the console interaction works better.

[afbjorklund]

So lima "works", and qemu "works". Left to do is making them work together, and add some documentation. 😃

Accelerator: https://docs.microsoft.com/en-us/virtualization/api/hypervisor-platform/hypervisor-platform (WHPX)

WSL is good but very slow in most cases , i prefer to have this in windows , :)

The main difference between WSL2 and Lima, is that lima uses a new virtual machine for each instance...
With the Windows Subsystem for Linux, all the system containers share the same VM kernel (like in LXC)

It is possible to start one Linux distribution (like Alpine), and then start system containers for Ubuntu or whatever.
Then the experience should be similar, same goes with sharing files - if opting in to use 9p (same as WSL uses)

[afbjorklund]

Almost got it to run, final hurdle is converting paths for qemu (dos, argh) and scripts for ssh (don't ask)

"[hostagent] qemu[stderr]: C:\\Program Files\\qemu\\qemu-system-x86_64.exe: cannot create PID file: Failed to create PID file"
"[hostagent] stdout=\"\", stderr=\"command-line line 0: invalid quotes\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"command-line line 0: invalid quotes\\r\\n\": exit status 255"

Fixes (PRs):

• lima compiles for GOOS=windows, cross-compiled on linux
• unittests runs for GOOS=windows, using wine64 on linux

Verified:

• regular limactl.exe operations (download, etc) works ok on Windows 10
• starting virtual machine with hardware acceleration works on Windows 10

Fallbacks:

• fallback to user "lima" using existing code, due to DOMAIN\user
• use id -u and id -g where available, otherwise fallback uid gid
• add home directory to the LimaUser, instead of using it "raw"
• use cygpath $HOME where available, otherwise just use "filepath"
• use windows paths (filepath) for host home and unix paths (path) for guest home
  • Expand tilde using the current home path #954

Workarounds:

• use tcp sockets instead of unix sockets, for hostagent/guestagent (needs tls)
• use named pipes instead of unix sockets, for qemu communication (needs patch)
• use unix paths for iso9660, since go-embed hardcodes slashes in filenames
  • Use forward slash for iso9660.Create on windows #953

User needs to add qemu, and regular tools - either MSYS2 or Git for Windows (MinGW) would work...

It's all normal programs, so it would be possible to install qemu-system-x86_64.exe and ssh.exe etc.
It does not require a Unix environment (like Cygwin) or other emulator, besides the regular QEMU (and Lima).

Using the "whpx" accelerator requires Windows with Hyper-V (Pro?), falling back to "haxm" would be possible.

---

Will make a PR for the fallbacks, but the rest needs a design decision - or to wait for AF_UNIX support ?

At this point it is just a proof-of-concept or technical demo, users are still recommended to use WSL2.

Note: this does not improve the performance (with Hyper-V), but it should be on par with the Mac version ?

I assume that all developers will be using Unix, and will not set up anything for PowerShell or DOS etc.

[arixmkii]

AF_UNIX support ?

@afbjorklund are you aware if there is any activity enabling AF_UNIX for windows builds on qemu side? This could benefit other projects as well. Like podman providing podman machine with MacOS like behavior instead as an alternative to WSL2 option.

[afbjorklund]

Sorry, I don't know anything about it. The information I stumbled upon so far looked more like "gross hacks" than anything else.

https://cygwin.com/pipermail/cygwin/2020-June/245088.html

https://stackoverflow.com/questions/23086038/what-mechanism-is-used-by-msys-cygwin-to-emulate-unix-domain-sockets

I will assume that Unix sockets are unavailable on Windows

[afbjorklund]

Afaik, Podman only uses Unix sockets for legacy (pre 18.09) Docker clients ? The other clients use SSH directly

[arixmkii]

Podman machine uses unix socket for qmp at least

-qmp unix://var/folders/<redacted>/T/podman/qmp_podman-machine-default.sock,server=on,wait=off 

And looks like for virtio-serial device

-device virtio-serial -chardev socket,path=/var/folders/<redacted>/T/podman/podman-machine-default_ready.sock,server=on,wait=off,id=podman-machine-default_ready 

These are extracts from podman machine start command line running on MacOS.