Add `podman run --bootc-vm` or so

[cgwalters]

Moving this from #95 (comment)

We had a thought a while ago that the functionality from podman-bootc might move to podman. However that has a large scope.

But what would have a much smaller and more obvious scope/match is moving just this functionality to podman:

Run a container that has a kernel embedded (in /usr/lib/modules) as a VM via direct boot (no bootloader) using the container's rootfs via virtiofs. The tool should support overriding init or just leave it with the default (which should work with systemd)

Strawman: podman run --bootc-vm <image>

cc @alicefr

Maybe it shouldn't be --bootc-vm but a more generic name; again the key idea is that we're running the kernel from the target image, and we're also just default to the commandline from the target image (e.g. systemd) - unlike e.g. kata we wouldn't have a runtime embedded.

Details

A really key question is whether the result of the execution of this should itself be represented as a podman container. This question is entangled with questions that arose in the original issue of whether we should require a virt stack on the host (as podman machine does) or whether it should come from a container.

If it does not appear as a (podman) container then perhaps it shouldn't be under podman run, but something more like podman machine runc or so?

[cgwalters]

This would force a bit of generalization of podman's virtualization abstraction that is right now only used by podman-machine, but I think it makes sense.

[cgwalters]

(A detail that would make everything a lot more streamlined here is if we changed podman-machine (at least on Linux) to always run this way - instead of fetching disk images as OCI artifacts, we just fetch a bootc container! Only specific directories would persist (such as /var/lib/containers); those directories could persist on the host or in attached volumes. That'd be a huge change of course though)

[alicefr]

@cgwalters is there a place where this should be broadly discussed?Any community meetings? Bootc meeting perhaps?

[cgwalters]

For both podman and bootc there are community meetings for sure. Bootc's are listed at https://github.com/bootc-dev/bootc?tab=readme-ov-file#community-discussion and podman's at https://podman.io/community

That said let's gather some initial thoughts from maintainers here cc @baude

---

In the short term though what would probably make sense is to just implement this core functionality as podman-bootc run-ephemeral or so here (and then we'd be able to use it for doing a persistent install).

[ericcurtin]

I think one of the tricky parts here is macOS... But still doable as part of podman remote there

[cgwalters]

I think one of the tricky parts here is macOS...

Yes it's hugely tricky in this case where the container files live inside the podman-machine VM. We don't want to do a copy - what we want is for a virtiofs mount for one VM to be backed by files stored in another. I don't think that's possible using the high level Virtualization API which just exposes the ability to mount host volumes. I guess it doesn't have to be a virtiofs mount, we could mount via e.g. NFS (over vsock e.g.) or so.

[Luap99]

I didn't read carefully the history but my initial thought process is that if you like to run any bootc image as VM this sounds to me more like like a new "runtime", e.g. there is --runtime krun currently to run an image as a VM via libkrun. That AFAIK is implemented via extension/plugin in crun. There is also https://github.com/containers/crun-vm/.

So I think you could have something like podman --runtime bootc run <bootc-image> design wise rather easily and the the details get abstracted in the external runtime. That would avoid the need for podman changes I guess.
cc @mheon @baude

[baude]

First, I think getting time at a Podman community meeting or cabal would be an appropriate forum. You would have most of the Podman maintainers available. I like this idea and topic and I am encouraged by it. I also see some possible benefits to Podman's workflow. Here are some of my additional thoughts:

• Paul is onto something but I would offer that we should have a default approach that just works and requires no extra knowledge.
• It makes sense somewhere in the podman machine subcommand context, though exactly where and how would be good design conversations.
• I am currently writing a Podman 6 HLD and will open this up for review very soon. Podman 6 is due in Spring of 2026. If we want this to be part of Podman 6, we need to get cracking to understand scope.

[cgwalters]

I didn't read carefully the history but my initial thought process is that if you like to run any bootc image as VM this sounds to me more like like a new "runtime", e.g. there is --runtime krun currently to run an image as a VM via libkrun. That AFAIK is implemented via extension/plugin in crun. There is also https://github.com/containers/crun-vm/.

Yes, I commented the same in #95 (comment)

I didn't read carefully the history but my initial thought process is that if you like to run any bootc image as VM this sounds to me more like like a new "runtime", e.g. there is --runtime krun currently to run an image as a VM via libkrun.

Right, and one of those details is that krun currently uses its own kernel, not the kernel from the image. That's possible to change of course with a different variant. I'll investigate krun a bit more again.

(From a quick glance at the source, is the status quo with podman-machine that it only does krun on MacOS, not Linux?)

---

I keep wavering a bit on whether we actually want this thing to be via podman run with all that implies - basically in showing up as a pod. Sometimes I might want that, other times not; at least the core low level primitive should work without being represented necessarily as a pod/container. (This topic of course gets to the interesting one that on Linux one could imagine that the podman-machine VM actually showed up as a visible pod when using the default connection...would that be useful? Maybe? There's similar implications for the use cases of this outside of podman machine)

[baude]

(From a quick glance at the source, is the status quo with podman-machine that it only does krun on MacOS, not Linux?)

Correct, we use libkrun only on Apple Silicon Macs.

[alicefr]

hi everyone, this discussion move in a different direction then what we have discussed #95. What the original PR covers is deploying an ephemeral rootless VM where we can build bootc images using podman from inside the VM to use all the privileged option and to be able to operate at disk level.

It relies on libvirt, so not calling directly QEMU, or libkrun. But for the rest it looks very close to what podman machine does.

In the original PR, the virt stack is deployed entirely in a container created by rootless podman, and then the bootc container is spawn inside the VM. The other options we were discussing in instead of having libvirt inside the container, to rely on libvirt from the host. My main question here is which one of the alternatives are the more desirable.

As Colin already mentioned, the goal is to boot from the artifacts from the bootc image.

IMO, a new runtime isn't necessary, since we aren't trying to launch containers, but only to create an ad-hoc environment for the bootc builds.

[cgwalters]

OK I really wanted to have this so I decided to spend a bit of time on it myself just to understand things better. There's a ~working PoC level result here: cgwalters/bootc-kit@ad5aaeb

A key way this works is basically via a podman run that mounts the host /usr in (so we have access to the virt stack from there), and just runs in the context of the target container image. Though note we also mount the target image again at /run/source-image so what we see is the pristine source image that we export via virtiofsd.

In this case it's important to understand that qemu and virtiofsd binaries come from the host, but everything is lifecycle bound to the container image (so yes, closer to crunvm...arguably we could put this there).

This is a really tiny amount of code in the end; yeah it's in Rust because well it's me writing it but eh, we can easily carry a Go implementation of this too.