kserve: load_model has no pool affinity — the predictor lands on any free GPU node, not the pool it was fitted for

[teemow]

Problem

With two GPU pools on one cluster (an L4 g6.xlarge pool and an L40S g6e.4xlarge pool), load_model for a 24 GB preset fitted against the L4 pool composed a predictor without any pool or accelerator affinity: the pod was scheduled onto the L40S node, the first GPU to appear. The L4 node never served; the model the person asked to run on the small pool consumed the large pool's card, and the model loaded against the L40S pool waited eight minutes for it to leave. The fit verdict named a pool and an instance type that the placement then ignored.

Proposed solution

The composed LLMInferenceService carries a node affinity for the pool the fit chose: the pool's own label (giantswarm.io/machine-pool=<name>) when the request named a pool or the fit selected one, else the accelerator label (nvidia.com/gpu.product or the pool's accelerator label) the preset's requirements imply. The verdict and the placement then agree; a model never lands on a card another pool was created for.

Acceptance criteria

• With two pools of different accelerators, a model fitted for the small pool runs on that pool's node (unit test on the composed spec; a serve on an installation).
• get_model names the pool the model is pinned to.

[teemow]

Mechanism, seen live the same evening: the cluster's second create_node_pool removes the serving slice's gpuPool.nodeSelector (cluster-manager's onlyPool is false with two pools), and every predictor composed afterwards renders nodeSelector: <none> — so with two pools nothing pins a model to the pool its fit chose. model-manager is the place to pin it per model; the slice-wide selector is by design only for the one-pool case.