Decimal bytes-to-MB conversion causes VM memory allocation inflation, risking host cgroup OOM kills

[Nachiket-Roy]

Summary

The OCI Runtime Specification defines container memory limits in bytes. When translating this value for most hypervisor backends, BytesToStringMB performs a decimal conversion (bytes / 1,000,000) instead of a binary conversion (bytes / (1024 * 1024)).

Since QEMU, Solo5, and Cloud Hypervisor interpret memory values in MiB, the resulting VM memory can exceed the original OCI byte limit.

Root cause

In pkg/unikontainers/hypervisors/utils.go:

func bytesToMB(bytes uint64) uint64 {
	const bytesInMB = 1000 * 1000
	return bytes / bytesInMB
}

func BytesToStringMB(argMem uint64) string {
	...
	userMem := bytesToMB(argMem)
	...
}

BytesToStringMB is used by the QEMU, Solo5 (hvt/spt), and Cloud Hypervisor backends when constructing their memory arguments.

However, these VMMs interpret memory values as binary MiB, not decimal MB.

Backend behavior

• QEMU (verified): Passing -m 256M results in exactly 268,435,456 bytes (256 MiB) of guest RAM.
• Solo5: Documentation indicates --mem is interpreted in MiB.
• Cloud Hypervisor: Documentation indicates size=<N>M is interpreted in MiB.
• Firecracker: Not affected, since firecracker.go already uses bytesToMiB (1024 * 1024).

Reproduction

1. Configure an OCI memory limit of 256,000,000 bytes.
2. Launch the container using the QEMU backend.
3. BytesToStringMB converts the value to 256.
4. urunc invokes QEMU with:

-m 256M

5. QEMU allocates 256 MiB (268,435,456 bytes) of guest RAM.

This can be verified from the guest boot memory map, where RAM ends at 0x10000000 (268,435,456 bytes).

Impact

For a configured limit of 256,000,000 bytes, the guest receives 268,435,456 bytes, approximately 12.4 MiB (~4.9%) more memory than requested.

As a result, the guest memory configured by the VMM no longer matches the OCI memory limit. Depending on how memory limits are enforced and the workload, this discrepancy may contribute to memory pressure or unexpected OOM behavior.

Suggested fix

Use binary conversion consistently for all hypervisor backends:

const bytesInMiB = 1024 * 1024

func bytesToMiB(bytes uint64) uint64 {
	return bytes / bytesInMiB
}

Then update BytesToStringMB (or rename it accordingly) to use this conversion.

Related issues

• Memory validation in BytesToStringMB fails to catch sub-viable values above the truncation threshold #819 - BytesToStringMB only guards against values that truncate to zero instead of enforcing a minimum usable memory.
• Sub-minimum memory requests fail via undocumented KVM crash or in-guest OOM panic rather than a clean error #820 - Low-memory guest failures are surfaced as generic VM exits without a clear runtime error.

Note: DefaultMemory = 256 remains correct after this change, since the supported hypervisors already interpret 256 as 256 MiB.

Environment

• urunc commit: 2785749875a1b13f543b6ad4137c68903a9ec87e
• QEMU: 8.2.2 (1:8.2.2+ds-0ubuntu1.17)
• Test image: harbor.nbfc.io/nubificus/urunc/redis-qemu-unikraft-initrd:latest

[cmainas]

Hello @Nachiket-Roy ,

thank you for reporting this issue. Indeed we should be careful with this convention. Looking at the runtime spec, I was not able to find any reference to MiB. However, since the memory field in the spec relates to cgroups (in Linux), the value we look at (limit) is expected to be in bytes and that is how we should treat it.

The problem comes with the monitors.

• In Solo5 the --mem argument seems to expect a value in MB

usage: solo5-hvt [ CORE OPTIONS ] [ MODULE OPTIONS ] [ -- ] KERNEL [ ARGS ]
KERNEL is the filename of the unikernel to run.
ARGS are optional arguments passed to the unikernel.
Core options:
  [ --mem=512 ] (guest memory in MB)
    --help (display this help)
    --version (display version information)
Compiled-in modules: core block net 
Compiled-in module options:
    --block:NAME=PATH (attach block device/file at PATH as block storage NAME)
  [ --block-sector-size:NAME=SECTORSIZE ] (set sector size for block device NAME; must be a power of two greater than or equal 512)
    --net:NAME=IFACE | @NN (attach tap at IFACE or at fd @NN as network NAME)
  [ --net-mac:NAME=HWADDR ] (set HWADDR for network NAME)

• Qemu most probably uses MiB, but the documentation is a bit weird From the help command:

-m [size=]megs[,slots=n,maxmem=size]
                configure guest RAM
                size: initial amount of guest memory
                slots: number of hotplug slots (default: none)
                maxmem: maximum amount of guest memory (default: none)
                Note: Some architectures might enforce a specific granularity

see "Note" and from the documentation

-m [size=]megs[,slots=n,maxmem=size]

    Sets guest startup RAM size to megs megabytes. Default is 128 MiB. Optionally, a suffix of “M” or “G” can be used to signify a value in megabytes or gigabytes respectively. Optional pair slots, maxmem could be used to set amount of hotpluggable memory slots and maximum amount of memory. Note that maxmem must be aligned to the page size.

    For example, the following command-line sets the guest startup RAM size to 1GiB, creates 3 slots to hotplug additional memory and sets the maximum memory the guest can reach to 4GiB:

    qemu-system-x86_64 -m 1G,slots=3,maxmem=4G

    If slots and maxmem are not specified, memory hotplug won’t be enabled and the guest startup RAM will never increase.

where the example clearly notes that 4G is 4GiB, but they call "G" gigbyte...

• Firecracker clearly specifies MiB https://github.com/firecracker-microvm/firecracker/blob/1faca2f70e7af6672fc7b4daacd9d037c874c335/docs/device-api.md?plain=1#L75
• Cloud hypervisor does not seem to specify if it uses MB or MiB in the documentation. Most probably it uses MiB.

In any case, we should convert the value to MiB instead of MB, as you suggest, by creating a similar function with BytesToStringMB that uses bytesToMiB. Also, for the monitors that use MB (solo5) we need to use MB, but without exceeding the limits.

[Nachiket-Roy]

Solo5 is consistent with QEMU, Firecracker, and Cloud Hypervisor here. It converts the supplied value to bytes using 1024 * 1024, i.e. MiB:

https://github.com/Solo5/solo5/blob/8b4a38cf8b03a26ea4a9b339ac5856c4ae3dde5b/tenders/hvt/hvt_main.c#L95

[cmainas]

Ok, then we can use MiB for all of them.