Skip to content

Add support for Linux memory policy #1282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 4, 2025
+168 −0
Merged

Add support for Linux memory policy #1282

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions config-linux.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -791,6 +791,45 @@ and may use a maximum memory bandwidth of 20% on socket 0 and 70% on socket 1.
}
```

## <a name="configLinuxMemoryPolicy" />Memory policy

**`memoryPolicy`** (object, OPTIONAL) sets the NUMA memory policy for the container.
For more information see the [set_mempolicy(2)][set_mempolicy.2] man page.

* **`mode`** *(string, REQUIRED)* -

A valid list of constants is shown below.

* `MPOL_DEFAULT`
* `MPOL_BIND`
* `MPOL_INTERLEAVE`
* `MPOL_WEIGHTED_INTERLEAVE`
* `MPOL_PREFERRED`
* `MPOL_PREFERRED_MANY`
* `MPOL_LOCAL`

* **`nodes`** *(string, REQUIRED)* - list of memory nodes from which nodemask is constructed to set_mempolicy(2). This is a comma-separated list, with dashes to represent ranges. For example, `0-3,7` represents memory nodes 0,1,2,3, and 7.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might not be 100% correct. nodes is not required for MPOL_DEFAULT or MPOL_LOCAL, in fact set_mempolicy(2) explicitly mention to set nodes to NULL, 0 (even if some variants can work as well as long as nodes is empty).

There is also a question about parsing nodes strings. libnuma supports more modes than the ones described here. Including "all" (all nodes), "+" for relative and "!" for negation. Wouldn´t it be better to align the implementations to configure mempolicy to a common format/method and provide a better user interface?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing out the issue in documentation, @fabbione! Fixed in PR #1294.

Regarding the convenience syntax for nodes, I was arguing against it in opencontainers/runc#4726 (comment). Let's continue the discussion there, if you disagree.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem at all, thanks for being so fast in fixing the docs etc.


* **`flags`** *(array of strings, OPTIONAL)* - list of flags to use with set_mempolicy(2).

A valid list of constants is shown below.

* `MPOL_F_NUMA_BALANCING`
* `MPOL_F_RELATIVE_NODES`
* `MPOL_F_STATIC_NODES`

### Example

```json
"linux": {
"memoryPolicy": {
"mode": "MPOL_INTERLEAVE",
"nodes": "2-3"
"flags": ["MPOL_F_STATIC_NODES"],
}
}
```

## <a name="configLinuxSysctl" />Sysctl

**`sysctl`** (object, OPTIONAL) allows kernel parameters to be modified at runtime for the container.
Expand Down Expand Up @@ -1073,6 +1112,7 @@ subset of the available options.
[tmpfs]: https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt

[full.4]: https://man7.org/linux/man-pages/man4/full.4.html
[set_mempolicy.2]: https://man7.org/linux/man-pages/man2/set_mempolicy.2.html
[mknod.1]: https://man7.org/linux/man-pages/man1/mknod.1.html
[mknod.2]: https://man7.org/linux/man-pages/man2/mknod.2.html
[namespaces.7_2]: https://man7.org/linux/man-pages/man7/namespaces.7.html
Expand Down
31 changes: 31 additions & 0 deletions features-linux.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,37 @@ Irrelevant to the availability of SELinux on the host operating system.
}
```

## <a name="linuxFeaturesMemoryPolicy" />MemoryPolicy

**`memoryPolicy`** (object, OPTIONAL) represents the runtime's implementation status of memoryPolicy.

* **`modes`** (array of strings, OPTIONAL). Recognized memory policies. Includes policies that may not be supported by the host operating system.
The runtime MUST recognize the elements in this array as the [`mode` of `linux.memoryPolicy` objects in `config.json`](config-linux.md#memory-policy).

* **`flags`** (array of strings, OPTIONAL). Recognized flags for memory policies. Includes flags that may not be supported by the host operating system.
The runtime MUST recognize the elements in this in the [`flags` property of the `linux.memoryPolicy` object in `config.json`](config-linux.md#memory-policy)

### Example

```json
"memoryPolicy": {
"modes": [
"MPOL_DEFAULT",
"MPOL_BIND",
"MPOL_INTERLEAVE",
"MPOL_WEIGHTED_INTERLEAVE",
"MPOL_PREFERRED",
"MPOL_PREFERRED_MANY",
"MPOL_LOCAL"
],
"flags": [
"MPOL_F_NUMA_BALANCING",
"MPOL_F_RELATIVE_NODES",
"MPOL_F_STATIC_NODES"
]
}
```

## <a name="linuxFeaturesIntelRdt" />Intel RDT

**`intelRdt`** (object, OPTIONAL) represents the runtime's implementation status of Intel RDT.
Expand Down
16 changes: 16 additions & 0 deletions features.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,22 @@ Here is a full example for reference.
"selinux": {
"enabled": true
},
"memoryPolicy": {
"modes": [
"MPOL_DEFAULT",
"MPOL_BIND",
"MPOL_INTERLEAVE",
"MPOL_WEIGHTED_INTERLEAVE",
"MPOL_PREFERRED",
"MPOL_PREFERRED_MANY",
"MPOL_LOCAL"
],
"flags": [
"MPOL_F_NUMA_BALANCING",
"MPOL_F_RELATIVE_NODES",
"MPOL_F_STATIC_NODES"
]
},
"intelRdt": {
"enabled": true
}
Expand Down
17 changes: 17 additions & 0 deletions schema/config-linux.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,23 @@
}
}
},
"memoryPolicy": {
"type": "object",
"properties": {
"mode": {
"$ref": "defs-linux.json#/definitions/MemoryPolicyMode"
},
"nodes": {
"type": "string"
},
"flags": {
"type": "array",
"items": {
"$ref": "defs-linux.json#/definitions/MemoryPolicyFlag"
}
}
}
},
"personality": {
"type": "object",
"$ref": "defs-linux.json#/definitions/Personality"
Expand Down
20 changes: 20 additions & 0 deletions schema/defs-linux.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,26 @@
"allow"
]
},
"MemoryPolicyMode": {
"type": "string",
"enum": [
"MPOL_DEFAULT",
"MPOL_BIND",
"MPOL_INTERLEAVE",
"MPOL_WEIGHTED_INTERLEAVE",
"MPOL_PREFERRED",
"MPOL_PREFERRED_MANY",
"MPOL_LOCAL"
]
},
"MemoryPolicyFlag": {
"type": "string",
"enum": [
"MPOL_F_NUMA_BALANCING",
"MPOL_F_RELATIVE_NODES",
"MPOL_F_STATIC_NODES"
]
},
"NetworkInterfacePriority": {
"type": "object",
"properties": {
Expand Down
35 changes: 35 additions & 0 deletions specs-go/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,8 @@ type Linux struct {
// IntelRdt contains Intel Resource Director Technology (RDT) information for
// handling resource constraints and monitoring metrics (e.g., L3 cache, memory bandwidth) for the container
IntelRdt *LinuxIntelRdt `json:"intelRdt,omitempty"`
// MemoryPolicy contains NUMA memory policy for the container.
MemoryPolicy *LinuxMemoryPolicy `json:"memoryPolicy,omitempty"`
// Personality contains configuration for the Linux personality syscall
Personality *LinuxPersonality `json:"personality,omitempty"`
// TimeOffsets specifies the offset for supporting time namespaces.
Expand Down Expand Up @@ -855,6 +857,19 @@ type LinuxIntelRdt struct {
EnableMBM bool `json:"enableMBM,omitempty"`
}

// LinuxMemoryPolicy represents input for the set_mempolicy syscall.
type LinuxMemoryPolicy struct {
// Mode for the set_mempolicy syscall.
Mode MemoryPolicyModeType `json:"mode"`

// Nodes representing the nodemask for the set_mempolicy syscall in comma separated ranges format.
// Format: "<node0>-<node1>,<node2>,<node3>-<node4>,..."
Nodes string `json:"nodes"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have been omitempty too?


// Flags for the set_mempolicy syscall.
Flags []MemoryPolicyFlagType `json:"flags,omitempty"`
}

// ZOS contains platform-specific configuration for z/OS based containers.
type ZOS struct {
// Namespaces contains the namespaces that are created and/or joined by the container
Expand Down Expand Up @@ -884,6 +899,26 @@ const (
ZOSUTSNamespace ZOSNamespaceType = "uts"
)

type MemoryPolicyModeType string

const (
MpolDefault MemoryPolicyModeType = "MPOL_DEFAULT"
MpolBind MemoryPolicyModeType = "MPOL_BIND"
MpolInterleave MemoryPolicyModeType = "MPOL_INTERLEAVE"
MpolWeightedInterleave MemoryPolicyModeType = "MPOL_WEIGHTED_INTERLEAVE"
MpolPreferred MemoryPolicyModeType = "MPOL_PREFERRED"
MpolPreferredMany MemoryPolicyModeType = "MPOL_PREFERRED_MANY"
MpolLocal MemoryPolicyModeType = "MPOL_LOCAL"
)

type MemoryPolicyFlagType string

const (
MpolFNumaBalancing MemoryPolicyFlagType = "MPOL_F_NUMA_BALANCING"
MpolFRelativeNodes MemoryPolicyFlagType = "MPOL_F_RELATIVE_NODES"
MpolFStaticNodes MemoryPolicyFlagType = "MPOL_F_STATIC_NODES"
)

// LinuxSchedulerPolicy represents different scheduling policies used with the Linux Scheduler
type LinuxSchedulerPolicy string

Expand Down
9 changes: 9 additions & 0 deletions specs-go/features/features.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ type Linux struct {
Apparmor *Apparmor `json:"apparmor,omitempty"`
Selinux *Selinux `json:"selinux,omitempty"`
IntelRdt *IntelRdt `json:"intelRdt,omitempty"`
MemoryPolicy *MemoryPolicy `json:"memoryPolicy,omitempty"`
MountExtensions *MountExtensions `json:"mountExtensions,omitempty"`
NetDevices *NetDevices `json:"netDevices,omitempty"`
}
Expand Down Expand Up @@ -132,6 +133,14 @@ type IntelRdt struct {
Enabled *bool `json:"enabled,omitempty"`
}

// MemoryPolicy represents the "memoryPolicy" field.
type MemoryPolicy struct {
// modes is the list of known memory policy modes, e.g., "MPOL_INTERLEAVE".
Modes []string `json:"modes,omitempty"`
// flags is the list of known memory policy mode flags, e.g., "MPOL_F_STATIC_NODES".
Flags []string `json:"flags,omitempty"`
}

// MountExtensions represents the "mountExtensions" field.
type MountExtensions struct {
// IDMap represents the status of idmap mounts support.
Expand Down
Loading