Suggestion Description
What happens
On our MI355X nodes with AMD Pensando ionic RoCE NICs, Mooncake's automatic GID selection picks a link-local address that cannot route between nodes. Cross-node KV transfer for prefill/decode disaggregation therefore does not work until we override the GID index by hand.
Setting these three variables makes it work on the same hardware and fabric, with nothing else changed:
MC_GID_INDEX=1
MC_IB_GID_INDEX=1
MOONCAKE_IB_GID_INDEX=1
Why it happens
Our ionic ports expose only IPv6 GIDs — there is no IPv4-mapped entry on any of them:
- index 0 is link-local (
fe80::/10), which only works on the local host
- index 1 is a routable ULA (
fc00::/7), which is the one we need
The Mooncake build shipped in the ATOM image looks for an IPv4-mapped GID first. When it finds none, it falls back to the first IPv6 GID that has a network device attached. Link-local GIDs do have a network device attached, so index 0 is selected and index 1 is never considered. There is no check for whether the address it picked can actually route off the host.
To see the decision being made, start the container with MC_LOG_LEVEL=INFO and look for Find best gid index: in the log.
Environment
- Image:
rocm/atom-dev@sha256:fb7825b54829a42b915d788d84eb2b1b8e6230cd549ef38d9262fd584518499e (created 2026-07-20)
- MI355X, AMD Pensando ionic RoCE NICs, IPv6-only RoCE fabric
- 2-node prefill/decode disaggregation with
kv_connector=mooncake
What we would like
Mooncake should pick a routable GID on ionic without a manual override, since a link-local address can never work for cross-node transfer.
Upstream kvcache-ai/Mooncake has since added a GID routability check (rdma_gid_probe.h, AutoGidCandidateClass), so updating the dependency may remove the need for the manual override. It is a large jump though: the fork this image builds from is pinned at b0b7c8c6c563 (2026-04-30) and upstream main is now 674 commits ahead of it, so there's real scope for regressions or incompatibilities with ATOM's integration. Worth noting the fork carries exactly one commit that isn't upstream (the ionic max_mr_size / max_res_rd_atom fix) which would need to be carried forward or confirmed unnecessary.
Scope
This only affects the Mooncake connector. MoRI-IO (kv_connector=moriio) handles GID selection itself and wants MORI_IB_GID_INDEX left unset, so the variables above should not be carried over to that path.
Operating System
No response
GPU
No response
ROCm Component
No response
Suggestion Description
What happens
On our MI355X nodes with AMD Pensando ionic RoCE NICs, Mooncake's automatic GID selection picks a link-local address that cannot route between nodes. Cross-node KV transfer for prefill/decode disaggregation therefore does not work until we override the GID index by hand.
Setting these three variables makes it work on the same hardware and fabric, with nothing else changed:
Why it happens
Our ionic ports expose only IPv6 GIDs — there is no IPv4-mapped entry on any of them:
fe80::/10), which only works on the local hostfc00::/7), which is the one we needThe Mooncake build shipped in the ATOM image looks for an IPv4-mapped GID first. When it finds none, it falls back to the first IPv6 GID that has a network device attached. Link-local GIDs do have a network device attached, so index 0 is selected and index 1 is never considered. There is no check for whether the address it picked can actually route off the host.
To see the decision being made, start the container with
MC_LOG_LEVEL=INFOand look forFind best gid index:in the log.Environment
rocm/atom-dev@sha256:fb7825b54829a42b915d788d84eb2b1b8e6230cd549ef38d9262fd584518499e(created 2026-07-20)kv_connector=mooncakeWhat we would like
Mooncake should pick a routable GID on ionic without a manual override, since a link-local address can never work for cross-node transfer.
Upstream
kvcache-ai/Mooncakehas since added a GID routability check (rdma_gid_probe.h,AutoGidCandidateClass), so updating the dependency may remove the need for the manual override. It is a large jump though: the fork this image builds from is pinned atb0b7c8c6c563(2026-04-30) and upstream main is now 674 commits ahead of it, so there's real scope for regressions or incompatibilities with ATOM's integration. Worth noting the fork carries exactly one commit that isn't upstream (the ionicmax_mr_size/max_res_rd_atomfix) which would need to be carried forward or confirmed unnecessary.Scope
This only affects the Mooncake connector. MoRI-IO (
kv_connector=moriio) handles GID selection itself and wantsMORI_IB_GID_INDEXleft unset, so the variables above should not be carried over to that path.Operating System
No response
GPU
No response
ROCm Component
No response