[QST]: Roadmap for adapting existing DRA CPU support to grouped mode

[JiaHanDong]

Hi maintainers,

First of all, thank you for the work on slurm-bridge. We would like to understand the future direction of the integration between slurm-bridge and the DRA CPU Driver.

Recently, we asked the DRA CPU driver community about allocating more than 32 CPU cores with the driver. The feedback we received was that splitting the allocation into multiple ResourceClaims is currently the only known workaround in individual mode, but the DRA CPU driver maintainers also mentioned that individual mode is expected to be gradually deprecated, and that grouped mode is the preferred direction going forward.

This raises a few questions for slurm-bridge.

Background

Today, slurm-bridge works well with the DRA CPU driver in individual mode because each CPU is exposed as a separate DRA device. Each device carries attributes such as:

• dra.cpu/cpuID
• dra.cpu/coreID
• dra.cpu/socketID

This gives slurm-bridge enough information to map Slurm's abstract core allocation to Linux CPU IDs, and then generate CEL selectors that select the exact CPU devices allocated by Slurm.

However, in grouped + group-by=machine mode, the DRA CPU driver exposes the whole machine as one grouped device. This mode can still accept an explicit opaque cpuset, but that cpuset must be expressed using Linux logical CPU IDs.

The problem is that Slurm's core_bitmap is an abstract core bitmap. Its bit order is not necessarily the same as Linux logical CPU ID order. They may differ on systems with multiple sockets, NUMA topology, SMT/hyper-threading, non-trivial CPU enumeration order from BIOS/Linux, reserved/offline CPUs, or other topology differences. In individual mode, slurm-bridge can reconstruct this mapping from the per-CPU DRA device attributes. In grouped machine mode, those per-CPU attributes are no longer exposed through ResourceSlices, so slurm-bridge no longer has the same source of mapping information.

Questions

1. Given that the DRA CPU driver maintainers are moving away from individual mode, does slurm-bridge have a planned direction for supporting grouped + group-by=machine mode?
2. Is there an intended way for slurm-bridge to convert Slurm's abstract core_bitmap into the Linux CPU ID based opaque cpuset required by the DRA CPU driver in grouped machine mode?
3. Would the recommended approach be for slurm-bridge to obtain or reconstruct a Slurm-equivalent CPU mapping on each node, or is there another design being considered?
4. Is there any public roadmap for DRA CPU support in slurm-bridge, especially regarding the transition from individual mode to grouped mode?

We are trying to understand the best direction before making local changes. The main concern is avoiding an implementation that depends heavily on individual mode if that mode is likely to become deprecated upstream.

Thanks in advance for any guidance.

[killianmuldoon]

Hi @JiaHanDong - thanks for bringing this up!

The 32 device limitation is a real sore point, and we've been working with the CPU DRA folks on a solution. slurm-bridge will adopt the CPU DRA grouped mode once the extrenal allocator and configuration is available. This work will be done as part of kubernetes-sigs/dra-driver-cpu#229.

We'll do the integration work as soon as that's merged. The hope is that it will be part of the next slurm-bridge minor release. I'll leave this issue open to track that change and integration.

[JiaHanDong]

Thank you for the clarification and for coordinating with the DRA CPU driver community. It is great to hear that slurm-bridge plans to adopt grouped mode once the external allocator and configuration work in kubernetes-sigs/dra-driver-cpu#229 is available. We will follow the progress of this issue and the next slurm-bridge minor release.

As a related roadmap question, does slurm-bridge have any near-term plans to support GPU partitioning or sharing—for example, MIG, vGPU, time-slicing, or other mechanisms that expose one physical GPU as multiple allocatable resources?

Thanks again for your guidance and work on this integration.

[killianmuldoon]

I don't think there's anything for the next release related to MIG and GPU partitioning. Could you create a separate issue for that? It would help us track and prioritize it internally.

[JiaHanDong]

Thanks! I’ve opened a separate issue: #41.