install.bat succeeds but RTX 5090 fails to run due to CUDA 12.4 sm_120 incompatibility

[Lv255]

Hi, first of all, thank you for sharing this great project.
I tried setting it up on Windows using the provided install.bat file — the installation went smoothly and all dependencies and .whl packages were installed successfully.

However, when running the project on an RTX 5090, it fails to start.
After checking, it seems that the issue comes from the fact that CUDA 12.4 does not support sm_120 architecture, which the 5090 requires.

I believe the .whl files included in the installer were built with CUDA 12.4, so newer GPUs (like the RTX 5090) can’t execute them properly.

Steps to Reproduce

1. Run install.bat on Windows
2. Try launching the app using an RTX 5090 GPU

Expected Result

The application runs normally on RTX 5090.

Actual Result

The app fails to start due to unsupported sm_120 architecture in the current CUDA version.

Suggestion

Updating the .whl packages to CUDA 12.8 or higher should solve the problem, since CUDA 12.8 officially adds support for sm_120.
It might also help to auto-detect GPU architecture during installation and fetch a compatible version accordingly.

Environment

• OS: Windows 11
• GPU: RTX 5090
• CUDA: 12.4 (installed automatically by install.bat)
• Proposed CUDA: 12.8 or above

[FishWoWater]

Hi, I think the following snippet should work with CUDA 12.8 (install_trellis.bat)

@echo off

echo *** Installing requirements
pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128
pip install xformers==0.0.28.post3 --index-url=https://download.pytorch.org/whl/cu128
pip install pillow imageio imageio-ffmpeg tqdm easydict opencv-python-headless scipy ninja rembg onnxruntime trimesh xatlas pyvista pymeshfix igraph transformers
pip install git+https://github.com/EasternJournalist/utils3d.git@9a4eb15e4021b67b12c460c7057d642626897ec8
pip install https://github.com/kingbri1/flash-attention/releases/download/v2.7.4.post1/flash_attn-2.7.4.post1+cu128torch2.7.0cxx11abiFALSE-cp310-cp310-win_amd64.whl
pip install kaolin -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.7.0_cu128.html

mkdir tmp 
git clone https://github.com/NVlabs/nvdiffrast.git ./tmp/extensions/nvdiffrast
pip install ./tmp/extensions/nvdiffrast
git clone --recurse-submodules https://github.com/JeffreyXiang/diffoctreerast.git ./tmp/extensions/diffoctreerast
pip install ./tmp/extensions/diffoctreerast
git clone https://github.com/autonomousvision/mip-splatting.git ./tmp/extensions/mip-splatting
pip install ./tmp/extensions/mip-splatting/submodules/diff-gaussian-rasterization/

pip install spconv-cu120
pip install gradio==4.44.1 gradio_litmodel3d==0.0.1

echo *** Finished TRELLIS install
echo.
echo *** Scroll up and check for errors. Do not assume it worked.
pause

Besides, replace https://github.com/FishWoWater/3DAIGC-API/blob/f8f6d96234ffa06758cc15e88023891ad0fc27a3/scripts/install.bat#L69C49-L69C96 with https://[data.pyg.org/whl/torch-2.7.0+cu128.html](https://data.pyg.org/whl/torch-2.7.0+cu128.html)