Sync benchmark env to PyTorch 2.13 to match fvdb-core build - #318
Merged
Conversation
The nightly benchmarks began failing on 2026-08-20 with ImportError: .../site-packages/fvdb/libfvdb.so: undefined symbol: _ZN3c1010ValueErrorC1ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE (c10::ValueError::ValueError(c10::SourceLocation, std::string)) when frgs imports fvdb, reported under the "Download mipnerf360 dataset" step. The nightly builds the fvdb-core wheel using fvdb-core's env/build_environment.yml and installs it into the env defined here. Those two pins had drifted: fvdb-core moved to pytorch-gpu=2.13.0 in "PyTorch 2.13" (openvdb/fvdb-core#738, 2026-08-20) while this env still pinned 2.11.0, so a wheel built against libtorch 2.13 was loaded against libtorch 2.11. Bump to 2.13.0 to match. Verified the full env solves on linux-64 with cuda-version=13.0: pytorch/libtorch 2.13.0 cuda130, torchvision 0.28.0, and the existing pycolmap<4 bound still resolving to 3.13.0. This is the same drift that required 6c2ede6 for the 2.10 -> 2.11 bump, so add a comment recording that the two files must move together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Mark Harris <mharris@nvidia.com>
harrism
requested review from
matthewdcong and
sifakis
and removed request for
a team
August 20, 2026 23:02
swahtz
approved these changes
Aug 21, 2026
This was referenced Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Nightly benchmarks have failed since 2026-08-20 (run 32362276082). Both benchmark jobs fail under the step named "Download mipnerf360 dataset", but the download isn't what breaks — that step invokes
frgs, which imports fVDB:Demangled:
c10::ValueError::ValueError(c10::SourceLocation, std::string)— a libtorch symbol.Root cause
The nightly builds the fvdb-core wheel in one environment and installs it into another, and the PyTorch pins had drifted apart:
fvdb-core/env/build_environment.ymltests/benchmarks/comparative/docker/benchmark_environment.ymlfvdb-core landed PyTorch 2.13 (openvdb/fvdb-core#738) on 2026-08-20, bumping its build env from 2.11.0 to 2.13.0. This env was not updated, so
fvdb_core-0.6.0.dev0-cp312-cp312-linux_x86_64.whlwas compiled against libtorch 2.13 and then loaded against libtorch 2.11. The Aug 19 nightly passed; Aug 20 was the first failure.Fix
Bump
pytorch-gputo 2.13.0 to match fvdb-core.Verified by solving the full environment (linux-64,
CONDA_OVERRIDE_CUDA=13.0) with only this line changed:torchvisionfollows to 0.28.0 automatically,cuda-version=13.0is unchanged, and the existingpycolmap<4bound still resolves to 3.13.0 — so this change is isolated to the PyTorch stack.Recurrence
This is the third time these two files have drifted: 1b6956f (2.8→2.10), 6c2ede6 (2.10→2.11), and now 2.11→2.13. I've added a comment on the pin recording that the two must move together, but a comment is not a guard — worth considering a CI check that compares the two pins, or having the fvdb-core torch bump update both. Happy to file a follow-up issue if useful.
Testing
The real verification is a nightly benchmark run, which this PR's checks don't exercise. Recommend a manual workflow dispatch after merge rather than waiting for the schedule.
🤖 Generated with Claude Code