Skip to content

BUILD.md

Latest commit

 

History

History
322 lines (234 loc) · 10.6 KB

BUILD.md

File metadata and controls

322 lines (234 loc) · 10.6 KB

Building Rerun

This is a guide to how to build Rerun.

See also

Getting started with the repository

First, install the Rust toolchain using the installer from https://rustup.rs/.

Then, clone the repository:

git clone [email protected]:rerun-io/rerun.git
cd rerun

Now install the pixi package manager: https://github.com/prefix-dev/pixi?tab=readme-ov-file#installation

Make sure cargo --version prints 1.90.0 once you are done.

Python/PyO3 configuration (important!)

This workspace uses PyO3 for Python bindings. Many crates have a transitive dependency on pyo3-build-config, which requires a configuration file to be present.

If you see this error:

error: failed to parse contents of PYO3_CONFIG_FILE
caused by:
  - 0: failed to open PyO3 config file at /path/to/rerun/rerun_py/pyo3-build.cfg
  - 1: No such file or directory (os error 2)

Run pixi run ensure-pyo3-build-cfg to generate the config file:

This file is normally generated automatically by pixi activation scripts, but if you're running cargo directly outside of pixi, you'll need to generate it first. The configuration is referenced in .cargo/config.toml:

PYO3_CONFIG_FILE = { value = "rerun_py/pyo3-build.cfg", relative = true }

For more details, see Python build configuration below.

If you are using an Apple-silicon Mac (M1, M2), make sure rustc -vV outputs host: aarch64-apple-darwin. If not, this should fix it:

rustup set default-host aarch64-apple-darwin && rustup install 1.90.0

Git-lfs

We use git-lfs to store big files in the repository, such as UI test snapshots. We aim to keep this project buildable without the need of git-lfs (for example, icons and similar assets are checked in to the repo as regular files). However, git-lfs is generally required for a proper development environment, e.g. to run tests.

Setting up git-lfs

The TL;DR is to install git-lfs via your favorite package manager (apt, Homebrew, MacPorts, etc.) and run git lfs install. See the many resources available online more details.

You can ensure that everything is correctly installed by running git lfs ls-files from the repository root. It should list some test snapshot files.

Useful git-lfs commands

# Install git-lfs in the repo (installs git hooks)
git lfs install

# Move a file to git lfs
git lfs track "path/to/file/or/pattern" # OR manually edit .gitattributes
git add --renormalize . # Moves already added files to lfs (according to .gitattributes)

# Move a file from lfs to regular git
git lfs untrack "path/to/file/or/pattern" # OR manually edit .gitattributes
git add --renormalize . # Moves already added files to regular git (according to .gitattributes)

# Push to a contributor remote (see https://github.com/cli/cli/discussions/8794#discussioncomment-8695076)
git push --no-verify

# Push git lfs files to contributor remote:
git push origin $(git branch --show-current) && git push --no-verify && git push origin --delete $(git branch --show-current)

See also this section in the egui docs.

Validating your environment

You can validate your environment is set up correctly by running:

pixi run check-env

Building and running the Viewer

Use this command for building and running the viewer:

pixi run rerun

Running the Rust examples

All Rust examples are set up as separate executables, so they can be run by specifying the corresponding package, for example:

cargo run -p dna

They will either connect to an already running rerun viewer, or spawn a new one. In debug builds, it will spawn target/debug/rerun if it exists, otherwise look for rerun on PATH.

Building and installing the Rerun Python SDK

Rerun is available as a package on PyPi and can be installed with pip install rerun-sdk.

Additionally, nightly dev wheels from head of main are available at https://github.com/rerun-io/rerun/releases/tag/prerelease.

Building from source

If you want to build from source, you can do so easily in the Pixi environment:

pixi run py-build

Or to create a wheel:

pixi run py-build-wheel

You can run scripts that depend on rerun within the uv environment. For example:

pixi run uv run examples/python/minimal/minimal.py`

Running the Python examples

You can also install all rerun example and their dependencies into the same uv environment using:

pixi run py-build-examples

Each example is installed as a target within the uv environment. For example:

pixi run uv run plots

Tests & tooling

# Run the unit tests
pixi run py-test

# Run the linting checks
pixi run py-lint

# Run the formatter
pixi run py-fmt

See also TESTING.md for an overview of our testing infrastructure.

Building an installable Python wheel

The py-build-wheels-sdk-only command builds a whl file:

pixi run py-build-wheels-sdk-only

Which you can then install in your own Python environment:

pip install ./dist/CURRENT_ARCHITECTURE/*.whl

IMPORTANT: unlike the official wheels, wheels produced by this method do not contain the viewer, so they may only be used for logging purposes.

Building and installing the Rerun C++ SDK

On Windows you have to have a system install of Visual Studio 2022 in order to compile the SDK and samples.

All other dependencies are downloaded by Pixi! You can run tests with:

pixi run -e cpp cpp-test

and build all C++ artifacts with:

pixi run -e cpp cpp-build-all

The Pixi build commands export a compile_commands.json compilation database to the build directory. This can be useful for developer tools, e.g. for setting up IntelliSense in VSCode.

Building the docs

High-level documentation for Rerun can be found at http://rerun.io/docs. It is built from the separate repository rerun-docs.

Building for the web

If you want to build a standalone Rerun executable that contains the web-viewer and a gRPC server, you need to install the wasm32-unknown-unknown Rust target and ensure the web_viewer feature flag is set when building rerun. This is automatically done by this shortcut which builds & runs the web viewer:

pixi run rerun-web

If you're on Windows you have to make sure that your git client creates symlinks, otherwise you may get errors during the build. Run git config --show-scope --show-origin core.symlinks to check if symlinks are enabled. You may need to turn on Windows developer mode in order to give the mklink command sufficient permissions. See also this Stack Overflow reply on the issue.

Improving compile times

As of today, we link everything statically in both debug and release builds, which makes custom linkers and split debuginfo the two most impactful tools we have at our disposal in order to improve compile times.

These tools can be configured through your Cargo configuration, available at $HOME/.cargo/config.toml.

macOS

On x64 macOS, use the zld linker and keep debuginfo in a single separate file.

Pre-requisites:

  • Install zld: brew install michaeleisel/zld/zld.

config.toml (x64):

[target.x86_64-apple-darwin]
rustflags = [
    "-C",
    "link-arg=-fuse-ld=/usr/local/bin/zld",
    "-C",
    "split-debuginfo=packed",
]

On Apple-silicon Mac (M1, M2), the default settings are already pretty good. The default linker is just as good as zld. Do NOT set split-debuginfo=packed, as that will make linking a lot slower. You can set split-debuginfo=unpacked for a small improvement.

config.toml (M1, M2):

[target.aarch64-apple-darwin]
rustflags = [
    "-C",
    "split-debuginfo=unpacked",
]

Linux

On Linux, use the mold linker and keep DWARF debuginfo in separate files.

Pre-requisites:

  • Install mold through your package manager.

config.toml:

[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = [
    "-C",
    "link-arg=-fuse-ld=/usr/bin/mold",
    "-C",
    "split-debuginfo=unpacked",
]

Windows

On Windows, use LLVM's lld linker and keep debuginfo in a single separate file.

Pre-requisites:

  • Install lld:
cargo install -f cargo-binutils
rustup component add llvm-tools-preview

config.toml:

[target.x86_64-pc-windows-msvc]
linker = "rust-lld.exe"
rustflags = [
    "-C",
    "split-debuginfo=packed",
]

Python build configuration (automatic)

The repository is configured to automatically generate a PYO3_CONFIG_FILE for stable Python builds. This file (rerun_py/pyo3-build.cfg) is automatically created when you first run any pixi run command, and ensures consistent cargo caching whether you build via pixi run py-build, uv sync --package rerun-sdk, or other methods.

The configuration is referenced in .cargo/config.toml:

PYO3_CONFIG_FILE = { value = "rerun_py/pyo3-build.cfg", relative = true }

If you need to regenerate this file (e.g., after changing Python versions), run:

pixi run ensure-pyo3-build-cfg

To inspect what configuration pyo3 is using, you can run:

PYO3_PRINT_CONFIG=1 pixi run py-build

For more details on pyo3 build configuration, see the PyO3 Building and Distribution documentation.