Add DGX Spark support and update documentation#5231
Open
zbrad wants to merge 38 commits into
Open
Conversation
Contributor
zbrad
commented
May 22, 2026
zbrad
commented
- Added FAISS_OUTPUT_NAME and FAISS_CUVS_SPARK_LIBRARY cmake options
- Added FAISS_C_OUTPUT_NAME cmake option in c_api
- Added scripts for building Spark library, Python bindings, and wheel packaging
- Updated branch changes summary and various documentation files
- Removed VS Code settings from repository
- Updated environment file and build scripts
Complete build system for FAISS-GPU wheels with CUDA 13.2 and Python 3.14
Added:
- Build scripts: build_{lib,pkg,wheel}.sh, package_wheel.sh, clean_build.sh
- Build configuration: Makefile, environment_cuda132_py314.yml
- Verification tool: verify_environment.py
- Documentation: QUICKSTART.md, BUILD_WHEEL_CUDA132.md, SETUP_COMPLETE.md
- Updated README.md with build instructions
Supports GPU architectures: 80 (A100), 86 (RTX3080), 89 (RTX4090), 90 (H100/DGX), 92 (RTX5090)
- Update default CUDA_ARCHS to include 92 (RTX 5090) - Add Blackwell details to GPU Architecture Codes section - Add GPU Support Summary section highlighting all generations - Update example commands to show full architecture list - Add Blackwell Architecture documentation link - Enhanced DGX Spark and latest GPU generation support
…ctions - Remove legacy Volta and Turing architecture support from all documentation - Add CUDA 13.2 architecture requirements note to README - Clarify that Ampere (8.0+) is minimum for this build - Update GPU Architecture Codes sections with generation labels - Add explanation for older GPU workaround (CUDA 12.x builds)
- Link to official NVIDIA CUDA 13.2 release notes - Link to deprecated architectures documentation - Clarify that Volta/Pascal removal is per NVIDIA CUDA 13.0+ policy
- DGX Spark is powered by NVIDIA GB10 Grace Blackwell superchip - Architecture 92 (Blackwell), not architecture 90 (Hopper) - Update documentation, examples, and GPU support matrix - Reference: https://marketplace.nvidia.com/en-us/enterprise/personal-ai-supercomputers/dgx-spark/
- Remove Volta (70) and Turing (75) from supported list in manifest - Add explicit 'NOT SUPPORTED' section explaining CUDA 13.0+ removal - Include link to NVIDIA CUDA 13.2 release notes on deprecated architectures - Update formatting and add Ada (89) for completeness - Users now clearly see why older GPUs aren't supported
- Remove Volta (70) and Turing (75) from available architectures list - Add explicit note that they are not supported in CUDA 13.2 - Add reference link to CUDA 13.2 release notes on deprecated architectures - Add GB10 (DGX Spark) to Blackwell (92) entry
- RTX 5090 = sm_120 (compute 12.0), NOT 92 - verified via nvidia-smi - DGX Spark GB10 = sm_100 (compute 10.0) - Turing (75) IS supported in CUDA 13.2 - only Maxwell/Pascal/Volta removed - Update all scripts, Makefile, docs, and examples - Default CUDA_ARCHS now: 75;80;86;89;90;100;120 - Verified with: nvcc --list-gpu-arch on CUDA 13.2
- build_lib_cuda132.sh: use OpenBLAS instead of Intel MKL (BLA_VENDOR) - build_lib_cuda132.sh: manually copy libfaiss_avx512.so after cmake install (cmake does not export faiss_avx512 as an installed target) - build_pkg_cuda132.sh: add CMAKE_SHARED_LINKER_FLAGS -L path so linker finds libfaiss_avx512.so when building swigfaiss_avx512 - package_wheel.sh: run auditwheel repair to bundle libfaiss*.so inside the wheel, excluding CUDA and OpenBLAS system libraries
- scripts/wsl/: env.sh, build.sh, verify.sh, check_wheel.py Complete WSL2 build pipeline helpers for CUDA 13.2 + Intel MKL 2025.3 - wsl_build.sh, test_install.sh: thin PowerShell-callable shims - fix_line_endings.ps1: Windows CRLF utility script - .gitattributes: enforce LF on all .sh / Makefile / .mk files - .gitignore: add build.log exclusion - Makefile: FAISS_VARIANT ?= gpu-cu132; quote CUDA_ARCHS in all recipes - package_wheel.sh: thread FAISS_VARIANT through to setup.py - build_lib_cuda132.sh: MKL_ROOT auto-detect, cuVS=OFF, CUDA PATH exports - build_pkg_cuda132.sh: MKL_ROOT in LD_LIBRARY_PATH - faiss/python/setup.py: dynamic name via FAISS_VARIANT env var (faiss-gpu-cu132, faiss-cpu, faiss-gpu, or plain faiss when unset) - docs/branches/faiss-gpu-cu132/: moved BUILD_WHEEL_CUDA132.md here; added QUICKSTART.md (WSL workflow), SETUP_COMPLETE.md, WHEEL_NAMING.md (ecosystem survey and naming rationale for faiss-gpu-cu132) - README.md: Quick Start updated for scripts/wsl/; docs list updated - QUICKSTART.md: stub pointer to docs/branches/faiss-gpu-cu132/
…CUDA 13.2 build - Add TEST_RESULTS.md: Complete test validation report (1,118 tests passed, 100% success) - Detailed breakdown of all test categories and results - GPU support validation and performance metrics - Build quality assessment and production readiness checklist - Add RELEASE_NOTES.md: GitHub binary release description - GPU architecture table with CUDA codes and SM codes (sm_75-sm_120) - GPU support by workload recommendations - Installation instructions and troubleshooting - Test results summary and feature validation - Build configuration and system requirements - Update QUICKSTART.md: Add links to release notes and test results - Update SETUP_COMPLETE.md: Reference new documentation files - Enhance docs/branches/faiss-gpu-cu132/QUICKSTART.md: - Expand GPU architecture table with new SM code column - Map CUDA codes to SM codes for clarity - Enhance docs/branches/faiss-gpu-cu132/SETUP_COMPLETE.md: - Add release notes reference in documentation section These changes provide comprehensive documentation for the FAISS-GPU CUDA 13.2 build release, including full test validation, GPU architecture support details, and installation/deployment guidance for users.
- Create automatic release notes workflow for GPU CUDA builds - Workflow triggers on GPU-tagged releases (v*-gpu-cu*) - Manually deployable via workflow_dispatch for existing tags - Automatically extracts RELEASE_NOTES.md into GitHub release body - Supports manual tag specification for retroactive release updates
merge updated main
- Move branch documentation to gpu-cu132/docs/ - Move WSL orchestration scripts to gpu-cu132/wsl/ - Move core build scripts to gpu-cu132/scripts/ - Update references across Makefile, README, docs, workflows (50+ path updates) - Fix SCRIPT_DIR path logic for deeper subdirectories - Correct relative links in relocated documentation - Make gpu-cu132 optional: default FAISS_VARIANT to empty (canonical FAISS) - Add configurable GPU_CU132_ROOT and GPU_CU132_SCRIPTS variables - Emphasize CUDA 13.2 toolkit is optional in README
Keep canonical versions in gpu-cu132/docs/ only: - Removed gpu-cu132/QUICKSTART.md (see gpu-cu132/docs/QUICKSTART.md) - Removed gpu-cu132/SETUP_COMPLETE.md (see gpu-cu132/docs/SETUP_COMPLETE.md)
Manual file manifest is redundant; kept information is already in README and docs.
- Default FAISS_ENABLE_CUVS to ON for cu132 script builds - Add Makefile FAISS_ENABLE_CUVS knob and pass-through to build targets - Include CONDA_PREFIX in CMAKE_PREFIX_PATH to help find_package(cuvs)
merge from main
- Added FAISS_OUTPUT_NAME and FAISS_CUVS_SPARK_LIBRARY cmake options - Added FAISS_C_OUTPUT_NAME cmake option in c_api - Added scripts for building Spark library, Python bindings, and wheel packaging - Updated branch changes summary and various documentation files - Removed VS Code settings from repository - Updated environment file and build scripts
Contributor
|
@zbrad It seems like there are a lot of cmake, binary files, GPU dependency that do not belong in the PR. Also we have a build-release workflow already that also has the GPU packages, you can refer to those for examples. |
Contributor
Author
|
Damn, I thought i got all the bad binaries, I'll rescrub. Sorry about that.
I did find a bug on the arm, which caused it to fail during build, but also was trying to build a spark only binary set, to minimize size and build time. I had put this project aside, but was now circling back to cleanup to contribute back.
…________________________________
From: Michael Norris ***@***.***>
Sent: Friday, May 22, 2026 4:36:48 PM
To: facebookresearch/faiss ***@***.***>
Cc: Brad Merrill ***@***.***>; Mention ***@***.***>
Subject: Re: [facebookresearch/faiss] Add DGX Spark support and update documentation (PR #5231)
[https://avatars.githubusercontent.com/u/102261413?s=20&v=4]mnorris11 left a comment (facebookresearch/faiss#5231)<#5231 (comment)>
@zbrad<https://github.com/zbrad>
Can you give an overview of the benefit of this support? We already build for up through blackwell, any reason Faiss GPU or cuVS-through-Faiss is not working on DGX Spark?
It seems like there are a lot of cmake, binary files, GPU dependency that do not belong in the PR. Also we have a build-release workflow already that also has the GPU packages, you can refer to those for examples.
—
Reply to this email directly, view it on GitHub<#5231 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAATEX5A5GDGE6KWKN4FB4L44DQBBAVCNFSM6AAAAACZJ4F5FCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DKMRTGM3DAMZYGI>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Contributor
|
@zbrad no worries, but before you do more work, is it the case that there was a gap in functionality, and the current Faiss GPU or cuVS-through-Faiss indeed does not work on DGX Spark platform? I have not tested it but it appears to run CUDA, so I thought Faiss would be working there. |
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…knob
Rename gpu-cu132/ -> gpu-cu/ (version-agnostic) and restructure the GPU wheel
build toolkit around explicit conventions:
- Build scripts named by CPU arch: *_x86_64.sh (Intel MKL, AVX2/AVX512) and
*_aarch64.sh (OpenBLAS + cuVS, SVE, DGX Spark SM 121).
- Single CUDA-version input via gpu-cu/scripts/cuda_env.sh (FAISS_CUDA_VER /
FAISS_CUDA_TAG; specify either, derive the other). CUDA_HOME auto-resolves to
/usr/local/cuda-<ver> on multi-toolkit hosts; warns on nvcc mismatch.
- Wheel package name faiss-gpu-${TAG}; CPU arch carried by the manylinux
platform tag, not the name. Single-GPU-arch builds append -sm<arch>
(e.g. faiss-gpu-cu132-sm121); multi-arch builds omit it.
- C++ libs libfaiss-{arch}-${TAG}.so via FAISS_OUTPUT_NAME / FAISS_C_OUTPUT_NAME.
- setup.py detects build arch from platform.machine(); Makefile gains
build-aarch64 / check-aarch64 and threads the version/arch knobs through.
Docs: split into BUILD_arch_x86_64.md (incl. WSL quick start) and
BUILD_arch_aarch64.md; rewrote WHEEL_NAMING.md and BRANCH_CHANGES_SUMMARY.md;
removed redundant QUICKSTART.md and stale duplicate trees
(docs/branches/, scripts/wsl/). Fixed SM-code errors (Spark is SM 121).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.