Nvidia GPU not detected on Windows 11

[BolverBlitz]

My System got
CPU: Ryzen 9 7950X3D + Integrated GPU (128GB)
GPU: RTX 3090 (24GB)

I see no manual config option, so i run it with python3 ./exo/main.py, it seems to work and downloads models.
But it claims it uses 128GB and 0TFLOPS so its not using the RTX Card.

I´ve also added CUDA=1 to my env as some people on linux sugested in issues $env:CUDA = "1"

nvcc --version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Apr_17_19:36:51_Pacific_Daylight_Time_2024
Cuda compilation tools, release 12.5, V12.5.40
Build cuda_12.5.r12.5/compiler.34177558_0

nvidia-smi

Wed Jan  8 16:31:11 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.94                 Driver Version: 560.94         CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090      WDDM  |   00000000:01:00.0  On |                  N/A |
|  0%   44C    P8             47W /  370W |    2744MiB /  24576MiB |     30%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A      6464    C+G   C:\Windows\explorer.exe                     N/A      |
|    0   N/A  N/A     11168    C+G   ...m Files\Mozilla Firefox\firefox.exe      N/A      |
|    0   N/A  N/A     13700    C+G   ...nt.CBS_cw5n1h2txyewy\SearchHost.exe      N/A      |
|    0   N/A  N/A     13736    C+G   ...2txyewy\StartMenuExperienceHost.exe      N/A      |
|    0   N/A  N/A     15680    C+G   ...Roaming\64Gram Desktop\Telegram.exe      N/A      |
|    0   N/A  N/A     15980    C+G   ...t.LockApp_cw5n1h2txyewy\LockApp.exe      N/A      |
|    0   N/A  N/A     17672    C+G   ...oration\NvContainer\nvcontainer.exe      N/A      |
|    0   N/A  N/A     20440    C+G   ...m Files\Mozilla Firefox\firefox.exe      N/A      |
|    0   N/A  N/A     21336    C+G   ...GeForce Experience\NVIDIA Share.exe      N/A      |
|    0   N/A  N/A     22320    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe      N/A      |
|    0   N/A  N/A     25152    C+G   ...es\Elgato\StreamDeck\StreamDeck.exe      N/A      |
|    0   N/A  N/A     27580    C+G   ...\AMD\CNext\CNext\RadeonSoftware.exe      N/A      |
|    0   N/A  N/A     27812    C+G   ...les\AMD\CNext\CNext\AMDRSSrcExt.exe      N/A      |
|    0   N/A  N/A     28540    C+G   ...les\Microsoft OneDrive\OneDrive.exe      N/A      |
|    0   N/A  N/A     28860    C+G   ...\cef\cef.win7x64\steamwebhelper.exe      N/A      |
|    0   N/A  N/A     30516    C+G   ...al\Discord\app-1.0.9175\Discord.exe      N/A      |
|    0   N/A  N/A     32904    C+G   ...\Programs\signal-desktop\Signal.exe      N/A      |
|    0   N/A  N/A     34100    C+G   ... Stream\101.0.3.0\GoogleDriveFS.exe      N/A      |
|    0   N/A  N/A     35300    C+G   ...38.0_x64__zpdnekdrzrea0\Spotify.exe      N/A      |
|    0   N/A  N/A     38268    C+G   ...tBakPCAgent\App\NetBak PC Agent.exe      N/A      |
|    0   N/A  N/A     67568    C+G   ...esktop\app-3.4.12\GitHubDesktop.exe      N/A      |
|    0   N/A  N/A     68620    C+G   ...Mozilla Thunderbird\thunderbird.exe      N/A      |
|    0   N/A  N/A     76684    C+G   ...5n1h2txyewy\ShellExperienceHost.exe      N/A      |
|    0   N/A  N/A     81580    C+G   ...crosoft\Edge\Application\msedge.exe      N/A      |
|    0   N/A  N/A     84284    C+G   ...__8wekyb3d8bbwe\WindowsTerminal.exe      N/A      |
+-----------------------------------------------------------------------------------------+

[RickyRAV]

hi there, an issue that i've had myself a lot of times is that sometimes the casing inside the exo/exo/topology/device_capabilities.py might not be the same as what gets detected.
For your case:
Your GPU name is: NVIDIA GeForce RTX 3090
but inside said file it's called:

"NVIDIA GEFORCE RTX 3090": DeviceFlops(fp32=35.6*TFLOPS, fp16=71.2*TFLOPS, int8=142.4*TFLOPS),
"NVIDIA GEFORCE RTX 3090 TI": DeviceFlops(fp32=40.0*TFLOPS, fp16=80.0*TFLOPS, int8=160.0*TFLOPS),
# RTX 20 series
"NVIDIA GEFORCE RTX 2060": DeviceFlops(fp32=6.45*TFLOPS, fp16=12.9*TFLOPS, int8=25.8*TFLOPS),

so maybe try to modify that file yourself, exo/exo/topology/device_capabilities.py LINE: 86 to
"NVIDIA GeForce RTX 3090": DeviceFlops(fp32=35.6*TFLOPS, fp16=71.2*TFLOPS, int8=142.4*TFLOPS)

[jli113]

same here on Ubuntu 24.04, CPU: Ryzen 9 5950X + Integrated GPU (128GB)
GPU: RTX 3090 Ti(24GB)