Unified storage

[cgwalters]

Status quo

See the filesystem docs.

Proposal 1 (Phase 0): optionally pull first into c/storage

This will probably want to start out as an opt-in experimental mode. Add some sort of config file or option that turns on mode for this (could live in the origin file for now I guess)

Basically our initial fetch we pull the base OS image via just directly doing a podman pull or skopeo copy the same way we pull LBIs.

From there, we do a reflink/hardlink copy into the ostree store (i.e. This would duplicate metadata, but not data on disk. Reflinks should always work. If we have to do hardlinks it could potentially confuse the st_nlink count if any LBIs happen to share layers. But that seems unlikely.

We'll also need to teach the garbage collection code about this.

Incremental transition

Probably the best way to do this is to change things so on upgrade, we only pull layers into the LBI storage if they didn't already exist in the ostree store or so.

More info

Today this project uses ostree-containers for the base image.

Much more recently, we landed logically bound app images which ended up requiring a bootc-owned containers/storage instance.

This left us in a confusing situation where we have two image storage backends used by bootc - one for the booted host, and one for LBIs.

The code they don't share is mostly containers/storage.

This issue tracks unification: Having bootc leverage more of containers/storage (which is going to likely require containers/storage improvements.

This issue also intersects #1016 in that we would need to get progress out of podman pulls

Note: I did a draft PR for this a while ago #215

Update 20250632: See also #1190 for transitioning to a "native composefs" storage.

[jlebon]

Would containers-storage be another potential backend? Some considerations there of course (mixing images of different trust level and duplication) but it seems natural to have the OS container go there just like any other image so then you'd manage it with podman as usual.

[cgwalters]

Potentially. That said, it was an intentional decision so far to separate application container storage from the OS. This ties into larger architectural decisions taken. Specifically today, we tend to view containers as optional. For example, we'd need to ensure things like podman image prune -a don't just blow away your OS too. I could imagine the OS being represented as an always-running container or something internally? But it needs careful design and thought.

[cheese]

How about a transactional-update/BTRFS backend? It is straightforward and naturally supports apply-live.

[vrothberg]

@cgwalters, do you want to track this separately from the podman backend?

[cgwalters]

@cgwalters, do you want to track this separately from the podman backend?

I've reworked this issue to now be about unified storage.

How about a transactional-update/BTRFS backend? It is straightforward and naturally supports apply-live.

Long story short, the goal here is to make composefs the technological heart of things. Depending on BTRFS specifically doesn't give us a lot of value versus what we already have, augmented with composefs. "Live apply" type things are related but distinct.

[cgwalters]

In the short term one thing we could do is basically add an opt-in flag (part of the origin? bootc.conf? something) that changes the flow for fetching the base image to:

• pull into bootc-owned c/storage (where LBIs go today)
• pull from that via ostree-container path into the ostree repo (this will suck, it will be a full physical copy for now) via the containers-storage: transport
• Then we optimize that

[cgwalters]

pull from that via ostree-container path into the ostree repo (this will suck, it will be a full physical copy for now) via the containers-storage: transport

Was thinking about this more and honestly we can probably rely on bypassing all the containers/storage/overlay abstractions as is done in https://github.com/containers/bootc/pull/215/files#diff-d61a00deb019ae32a7f695e1323119424ce1a268fb34d0639f3e74fb2ed78fedR244 - especially for a storage version we "own".

[cgwalters]

Just to cross-reference a few things, I also have been working in the background on https://github.com/cgwalters/composefs-oci which is basically a "clean slate experimental playground" looking at "what might it look like if we made composefs the heart of things" - which it basically isn't today for ostree or c/storage (it's kind of awkwardly glued on the side to both).

In reality there's a lot to do to productize that, and the most practical short term will likely continue to be improving c/storage and strategically "hollowing out" ostree where we can; xref ostreedev/ostree#3291