Nightly wheel RUNPATH doesn't reach libnvrtc.so.13: `import fvdb` fails without LD_LIBRARY_PATH workaround

[harrism]

Summary

The fvdb-core 0.5.0.dev*+pt###.cu### NVIDIA pip-nightly wheels don't link to libnvrtc.so.13 in a way that survives a vanilla import fvdb. End users hit

ImportError: libnvrtc.so.13: cannot open shared object file: No such file or directory

unless something else on the system has already prepended a directory containing libnvrtc.so.13 to LD_LIBRARY_PATH. In conda-on-Linux deployments that "something else" is typically the activation hook of an unrelated package (mkl-devel's mklvars.sh), which users don't realise they're depending on until they trim their env.

The conda-forge fvdb-core 0.4.2 stable package isn't affected — this is specific to the NVIDIA-built nightly wheels.

Symptom

_fvdb_cpp.so from the nightly wheel only has $ORIGIN on its RUNPATH:

$ readelf -d $CONDA_PREFIX/lib/python3.12/site-packages/fvdb/_fvdb_cpp.so \
    | grep -E 'RUNPATH|NEEDED.*nvrtc'
 0x000000000000001d (RUNPATH)  Library runpath: [$ORIGIN]
 0x0000000000000001 (NEEDED)   Shared library: [libnvrtc.so.13]

$ORIGIN resolves to <env>/lib/python3.12/site-packages/fvdb/, which doesn't contain libnvrtc.so.13. In a typical conda env the library lives at <env>/lib/libnvrtc.so.13 — three levels up from $ORIGIN, not reachable from the current RUNPATH.

Reproduction (~3 min, clean container)

docker run --rm nvidia/cuda:13.0.2-base-rockylinux8 bash -c '
dnf install -y git procps-ng findutils
curl -fsSL https://github.com/mamba-org/micromamba-releases/releases/latest/download/micromamba-linux-64 \
  -o /usr/local/bin/micromamba && chmod +x /usr/local/bin/micromamba
eval "$(/usr/local/bin/micromamba shell hook --shell bash)"
# Lean env: python + cuda-nvrtc + pip, no MKL.
micromamba create -y -n gf -c conda-forge python=3.12 cuda-nvrtc=13.* pip
micromamba activate gf
pip install --pre --no-deps "fvdb-core>=0.5.0.dev0" \
  --extra-index-url=https://d36m13axqqhiit.cloudfront.net/simple-nightly
python -c "import fvdb; print(fvdb.__version__)"
# → ImportError: libnvrtc.so.13: cannot open shared object file

LD_LIBRARY_PATH=$CONDA_PREFIX/lib python -c "import fvdb; print(fvdb.__version__)"
# → works
'

In a "heavy" conda env that pulls blas=*=mkl, mkl-devel's mklvars.sh prepends $CONDA_PREFIX/lib to LD_LIBRARY_PATH on activate, so the import succeeds without the explicit workaround — but only by accident, via an unrelated package's side effect.

Possible fixes

Listed roughly cheapest → most self-contained, for the maintainers to weigh:

1. Extend RUNPATH to reach the conda env's lib/ dir. Add $ORIGIN/../../.. to the wheel's RUNPATH at link time. Cheapest; works for the common conda + pip-nightly deployment.

2. Declare a nvidia-cuda-nvrtc-cu13 runtime dep and target its pip layout. Make the wheel's Requires-Dist include nvidia-cuda-nvrtc-cu13 (which installs libnvrtc.so.13 under <sp>/nvidia/cuda_nvrtc/lib) and add $ORIGIN/../nvidia/cuda_nvrtc/lib to RUNPATH. PyTorch and other CUDA-using wheels follow this pattern. Robust for pip-only environments; conda users still fall back to the conda libnvrtc via the option-1 RUNPATH if both are present.

3. Bundle libnvrtc.so.13 inside the wheel under fvdb/lib/ (auditwheel-style) and add $ORIGIN/lib to RUNPATH. Largest wheel (libnvrtc is ~50 MB) but the only option that's truly "pip install and it works" with no external library presence required.

(A __init__.py ctypes.CDLL("libnvrtc.so.13", mode=RTLD_GLOBAL) shim would also work but is fragile relative to the linker-level fixes above.)

Combining (1) + (2) is probably the best middle ground — covers conda + pip-nvidia-packages without the wheel-size cost of (3).

Refs

• Downstream issue documenting the workaround on the consumer side, with mklvars.sh traceback: https://github.com/openvdb/nanovdb-ghost-fluid/issues/9
• Nightly wheel index: https://d36m13axqqhiit.cloudfront.net/simple-nightly
• Wheel inspection one-liner: readelf -d $CONDA_PREFIX/lib/python*/site-packages/fvdb/_fvdb_cpp.so | grep -E 'RUNPATH|NEEDED.*nvrtc'