comfyui: torch-bin cannot be substituted, so CUDA means building torch from source

[rokokol]

Nixpkgs version

Unstable (26.11), 0e251e2

Describe the problem

pkgs.comfyui builds its Python environment itself, pinning torch's CUDA packages:

python = python3.override {
  self = python;
  packageOverrides = final: prev: {
    torch = prev.torch.override { cudaPackages = cudaPackages_13; };
    triton = prev.triton.override { cudaPackages = cudaPackages_13; };
  };
};

On a machine with an NVIDIA GPU this leaves no practical way to run it:

With the default torch there is no GPU support, and no CPU fallback either. python3Packages.torch is built with cudaSupport = false by default, and ComfyUI does not degrade to CPU on its own — in comfy/model_management.py (v0.31.1) cpu_state is only set to CPUState.CPU when --cpu is passed, otherwise get_torch_device() ends in torch.device(torch.cuda.current_device()). So the stock package aborts at startup unless the user knows to pass --cpu, which then pins them to CPU inference:

$ nix build -f . comfyui
$ ./result/bin/comfyui --quick-test-for-ci   # stock package, nothing overridden
[INFO] Found comfy_kitchen backend cuda: {'available': False, …}
Traceback (most recent call last):
  File "/nix/store/…-comfyui-0.31.1/share/comfyui/main.py", line 239, in <module>
    import execution
  File "/nix/store/…-comfyui-0.31.1/share/comfyui/execution.py", line 18, in <module>
    import comfy.model_management
  File "/nix/store/…-comfyui-0.31.1/share/comfyui/comfy/model_management.py", line 363, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
                                  ~~~~~~~~~~~~~~~~^^
  File "/nix/store/…-comfyui-0.31.1/share/comfyui/comfy/model_management.py", line 212, in get_torch_device
    return torch.device(torch.cuda.current_device())
                        ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/nix/store/…-python3-3.14.7-env/lib/python3.14/site-packages/torch/cuda/__init__.py", line 484, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
$ echo $?
1
$ ./result/bin/comfyui --cpu --quick-test-for-ci >/dev/null; echo $?
0

It fails at import time, at module scope, before the server starts — so with services.comfyui.enable = true and nothing else set, the unit crash-loops on Restart = "always". Note that the package's own installCheckPhase runs comfyui --cpu --quick-test-for-ci, so this path is never exercised by the check.

Turning cudaSupport on means compiling torch from source. The pin above is cudaPackages_13, while the default cudaPackages is currently 12.9, so the resulting torch does not match what any cache carries. For a single desktop GPU this is not a viable path.

And the prebuilt wheels cannot be substituted. torch-bin would solve it — it ships CUDA in the wheel and takes a cudaPackages argument, so the pin above would still apply to it. But it cannot be reached through .override, because the package replaces the caller's packageOverrides rather than composing with it:

$ nix eval --impure --expr '
let pkgs = import <nixpkgs> { };
    p1 = pkgs.python3.override { packageOverrides = f: p: { mySentinel = "caller"; }; };
    p2 = p1.override { packageOverrides = f: p: { packageSentinel = "package"; }; };
in { callerSurvives = p2.pkgs ? mySentinel; packageApplied = p2.pkgs ? packageSentinel; }'
{ callerSurvives = false; packageApplied = true; }

So

comfyui.override {
  python3 = python3.override {
    packageOverrides = _: p: {
      torch = p.torch-bin;
      torchvision = p.torchvision-bin;
      torchaudio = p.torchaudio-bin;
    };
  };
}

silently does nothing, and the only remaining option is a global overlay on python3Packages, which drags every other torch consumer on the system along with it.

Expected behaviour

Compose the package's overrides on top of the caller's instead of discarding them:

python = python3.override (old: {
  self = python;
  packageOverrides = lib.composeExtensions (old.packageOverrides or (_: _: { })) (
    final: prev: {
      torch = prev.torch.override { cudaPackages = cudaPackages_13; };
      triton = prev.triton.override { cudaPackages = cudaPackages_13; };
    }
  );
});

With that ordering the caller supplies the base package (torch-bin) and the package still applies its CUDA pin on top, so nothing about the current default changes. Running the sentinel check above against that shape gives { callerSurvives = true; packageApplied = true; }.

On top of that, a ready-made variant would spare every user the incantation, in the shape nixpkgs already uses for the same problem:

ollama-rocm = callPackage ../by-name/ol/ollama/package.nix { acceleration = "rocm"; };
ollama-cuda = callPackage ../by-name/ol/ollama/package.nix { acceleration = "cuda"; };

i.e. an argument on comfyui selecting where torch comes from, with the binary-wheel variant exposed as its own attribute.

Additional context

#550095 adds a module-level acceleration option that injects --cpu and flips cudaSupport. That fixes the flag half of the first point, and is a good change, but it does not make the prebuilt wheels reachable: cudaSupport = true still means a source build.

I am happy to open a PR for the composeExtensions change if that shape sounds right.

Notify maintainers

@caniko @SuperSandro2000

---

Assisted-by: Claude Code (claude-opus-5). The commands quoted above were run on my own machine and I have reviewed the report.

[SuperSandro2000]

Comfyui disables GPU support if it is not cuda 13, so that is required until we bump the default.

I compile it myself, takes roughly 2 hours.

What ollama does is annoying as the update bot always opens PRs for multiple packages even if they are truly the same and it also makes things harder as you need to add overrides to all dependencies which also need cuda.

Feel free to do a PR with lib.composeExtensions, that is indeed an oversight on my end.

[rokokol]

Thanks! I'll get started within the week.