Skip to content
This repository was archived by the owner on Feb 10, 2025. It is now read-only.

Commit d0b5be5

Browse files
OliEfrOliEfr
committed
initial commit
0 parents  commit d0b5be5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

61 files changed

+34588
-0
lines changed

‎.devcontainer/Dockerfile

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
FROM nvidia/cuda:12.6.3-devel-ubuntu22.04
2+
3+
# Prevent interactive prompts during package installation
4+
ENV DEBIAN_FRONTEND=noninteractive
5+
6+
# Install system dependencies
7+
RUN apt-get update && apt-get install -y --no-install-recommends \
8+
python3.11 \
9+
python3-pip \
10+
python3.11-venv \
11+
build-essential \
12+
git \
13+
curl \
14+
ssh \
15+
mesa-utils \
16+
&& rm -rf /var/lib/apt/lists/* && apt-get clean
17+
18+
# Create symbolic links for python
19+
RUN ln -sf /usr/bin/python3.11 /usr/bin/python \
20+
&& ln -sf /usr/bin/python3.11 /usr/bin/python3
21+
22+
# Create non-root user
23+
ARG USERNAME=vscode
24+
ARG USER_UID=1000
25+
ARG USER_GID=$USER_UID
26+
27+
RUN groupadd --gid $USER_GID $USERNAME \
28+
&& useradd --uid $USER_UID --gid $USER_GID -m $USERNAME
29+
30+
# Set environment variables for CUDA
31+
ENV PATH="/usr/local/cuda-12.0/bin:${PATH}"
32+
ENV LD_LIBRARY_PATH="/usr/local/cuda-12.0/lib64:${LD_LIBRARY_PATH}"
33+
34+
ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
35+
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics
36+
37+
# Create and set proper permissions for workspace directory
38+
RUN mkdir -p /workspace && chown -R $USERNAME:$USERNAME /workspace && chmod -R 755 /workspace
39+
40+
# Create virtual environment with proper permissions
41+
ENV VIRTUAL_ENV=/home/vscode/venv
42+
RUN python -m venv $VIRTUAL_ENV && \
43+
chown -R $USERNAME:$USERNAME $VIRTUAL_ENV
44+
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
45+
46+
# Switch to non-root user
47+
USER $USERNAME
48+
49+
# Install development tools
50+
RUN pip install --no-cache-dir ruff
51+
52+
# Create a temporary directory for installation with proper permissions
53+
WORKDIR /tmp/install
54+
55+
# Copy pyproject.toml with correct ownership
56+
COPY --chown=$USERNAME:$USERNAME pyproject.toml .
57+
58+
# Install dependencies as non-root user
59+
RUN pip install --no-cache-dir -e ".[test,gpu]"
60+
61+
# Set final working directory
62+
WORKDIR /workspace

‎.devcontainer/devcontainer.linux.json

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
{
2+
"name": "Crazyflow Development",
3+
"build": {
4+
"dockerfile": "Dockerfile",
5+
"context": ".."
6+
},
7+
"customizations": {
8+
"vscode": {
9+
"extensions": [
10+
"ms-python.python",
11+
"ms-python.pylance",
12+
"charliermarsh.ruff",
13+
"ms-vscode.test-adapter-converter",
14+
"ms-python.pytest-adapter",
15+
"nvidia.nsight-vscode-edition",
16+
"ms-azuretools.vscode-docker",
17+
"ms-toolsai.jupyter-renderers",
18+
"ms-toolsai.jupyter",
19+
"ms-toolsai.jupyter-keymap",
20+
"donjayamanne.vscode-default-python-kernel"
21+
22+
],
23+
"settings": {
24+
"python.defaultInterpreterPath": "/home/vscode/venv/bin/python",
25+
"jupyter.kernels.excludePythonEnvironments": [
26+
"/usr/bin/python",
27+
"/usr/bin/python3.10",
28+
"/usr/python3.10",
29+
"/usr/python3.10"
30+
], // hide unrequired python interpreters, so that the right one gets selected for jupyter https://github.com/microsoft/vscode/issues/130946#issuecomment-1899389049
31+
"python.testing.pytestEnabled": true, // test framework
32+
"python.testing.pytestArgs": [
33+
"-m",
34+
"not render"
35+
],
36+
"python.testing.unittestEnabled": false, // test framework
37+
"python.testing.nosetestsEnabled": false, // test framework
38+
"editor.formatOnSave": true,
39+
"editor.codeActionsOnSave": {
40+
"source.fixAll.ruff": true,
41+
"source.organizeImports.ruff": true
42+
},
43+
"[python]": {
44+
"editor.defaultFormatter": "charliermarsh.ruff",
45+
"editor.formatOnSave": true,
46+
"editor.rulers": [
47+
100
48+
]
49+
}
50+
}
51+
}
52+
},
53+
"containerEnv": {
54+
"DISPLAY": "${localEnv:DISPLAY}",
55+
"XAUTHORITY": "${localEnv:XAUTHORITY}"
56+
},
57+
"runArgs": [
58+
"--interactive",
59+
"--net=host",
60+
"--ipc=host",
61+
// "--gpus=all", //use only with GPU
62+
// "--runtime=nvidia" //use only with GPU
63+
],
64+
"postCreateCommand": "python -m pip install -e '.[test,gpu]'",
65+
"remoteUser": "vscode"
66+
}

‎.devcontainer/devcontainer.wsl2.json

+75
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
{
2+
"name": "Crazyflow Development",
3+
"build": {
4+
"dockerfile": "Dockerfile",
5+
"context": ".."
6+
},
7+
"customizations": {
8+
"vscode": {
9+
"extensions": [
10+
"ms-python.python",
11+
"ms-python.pylance",
12+
"charliermarsh.ruff",
13+
"ms-vscode.test-adapter-converter",
14+
"ms-python.pytest-adapter",
15+
"nvidia.nsight-vscode-edition",
16+
"ms-azuretools.vscode-docker",
17+
"ms-toolsai.jupyter-renderers",
18+
"ms-toolsai.jupyter",
19+
"ms-toolsai.jupyter-keymap",
20+
"donjayamanne.vscode-default-python-kernel"
21+
22+
],
23+
"settings": {
24+
"python.defaultInterpreterPath": "/home/vscode/venv/bin/python",
25+
"jupyter.kernels.excludePythonEnvironments": [
26+
"/usr/bin/python",
27+
"/usr/bin/python3.10",
28+
"/usr/python3.10",
29+
"/usr/python3.10"
30+
], // hide unrequired python interpreters, so that the right one gets selected for jupyter https://github.com/microsoft/vscode/issues/130946#issuecomment-1899389049
31+
"python.testing.pytestEnabled": true, // test framework
32+
"python.testing.pytestArgs": [
33+
"-m",
34+
"not render"
35+
],
36+
"python.testing.unittestEnabled": false, // test framework
37+
"python.testing.nosetestsEnabled": false, // test framework
38+
"editor.formatOnSave": true,
39+
"editor.codeActionsOnSave": {
40+
"source.fixAll.ruff": true,
41+
"source.organizeImports.ruff": true
42+
},
43+
"[python]": {
44+
"editor.defaultFormatter": "charliermarsh.ruff",
45+
"editor.formatOnSave": true,
46+
"editor.rulers": [
47+
100
48+
]
49+
}
50+
}
51+
}
52+
},
53+
// mounts required for WSL2 X11
54+
"mounts": [
55+
"source=/tmp/.X11-unix,target=/tmp/.X11-unix,type=bind",
56+
"source=/mnt/wslg,target=/mnt/wslg,type=bind",
57+
"source=/usr/lib/wsl,target=/usr/lib/wsl,type=bind"
58+
],
59+
"containerEnv": {
60+
"DISPLAY": "${localEnv:DISPLAY}",
61+
"XAUTHORITY": "${localEnv:XAUTHORITY}",
62+
"WAYLAND_DISPLAY": "${localEnv:WAYLAND_DISPLAY}", // WSL2
63+
"XDG_RUNTIME_DIR": "${localEnv:XDG_RUNTIME_DIR}", // WSL2
64+
"PULSE_SERVER": "${localEnv:PULSE_SERVER}" // WSL2
65+
},
66+
"runArgs": [
67+
"--interactive",
68+
"--net=host",
69+
"--ipc=host"
70+
// "--gpus=all", //use only with GPU
71+
// "--runtime=nvidia" //use only with GPU
72+
],
73+
"postCreateCommand": "python -m pip install -e '.[test,gpu]'",
74+
"remoteUser": "vscode"
75+
}

‎.github/workflows/ruff.yml

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
name: Ruff
2+
on: [ push, pull_request ]
3+
jobs:
4+
ruff:
5+
runs-on: ubuntu-latest
6+
steps:
7+
- uses: actions/checkout@v4
8+
- uses: astral-sh/ruff-action@v1

‎.github/workflows/testing.yml

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: Testing # Skips RL tests because stable-baselines3 comes with a lot of heavy-weight dependencies
2+
3+
on: [push]
4+
5+
jobs:
6+
test:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- uses: actions/checkout@v4
10+
- uses: mamba-org/setup-micromamba@v1
11+
with:
12+
micromamba-version: '2.0.2-1' # any version from https://github.com/mamba-org/micromamba-releases
13+
environment-name: test-env
14+
init-shell: bash
15+
create-args: python=3.11
16+
cache-environment: true
17+
- name: Install dependencies and package
18+
run: pip install .[test]
19+
shell: micromamba-shell {0}
20+
- name: Test with pytest
21+
run: pytest tests --cov=crazyflow
22+
shell: micromamba-shell {0}

‎.gitignore

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
**.egg-info/
2+
**/__pycache__/
3+
logs
4+
secrets
5+
**/*.secret
6+
saves
7+
.pytest_cache
8+
*.json
9+
**/*.csv
10+
build
11+
.venv
12+
!/.devcontainer/devcontainer.json
13+
!/.devcontainer/devcontainer.linux.json
14+
!/.devcontainer/devcontainer.wsl2.json
15+
!/.vscode/launch.json

‎.vscode/launch.json

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
// Use IntelliSense to learn about possible attributes.
3+
// Hover to view descriptions of existing attributes.
4+
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
5+
"version": "0.2.0",
6+
"configurations": [
7+
{
8+
"name": "Python Debugger: Current File",
9+
"type": "debugpy",
10+
"request": "launch",
11+
"program": "${file}",
12+
"console": "integratedTerminal",
13+
"justMyCode": false
14+
}
15+
]
16+
}

‎README.md

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# crazyflow
2+
Fast, parallelizable simulations of Crazyflies with JAX and MuJoCo.
3+
4+
[![Python Version]][Python Version URL] [![Ruff Check]][Ruff Check URL] [![Documentation Status]][Documentation Status URL] [![Tests]][Tests URL]
5+
6+
[Python Version]: https://img.shields.io/badge/python-3.10+-blue.svg
7+
[Python Version URL]: https://www.python.org
8+
9+
[Ruff Check]: https://github.com/utiasDSL/crazyflow/actions/workflows/ruff.yml/badge.svg?style=flat-square
10+
[Ruff Check URL]: https://github.com/utiasDSL/crazyflow/actions/workflows/ruff.yml
11+
12+
[Documentation Status]: https://readthedocs.org/projects/crazyflow/badge/?version=latest
13+
[Documentation Status URL]: https://crazyflow.readthedocs.io/en/latest/?badge=latest
14+
15+
[Tests]: https://github.com/utiasDSL/crazyflow/actions/workflows/testing.yml/badge.svg
16+
[Tests URL]: https://github.com/utiasDSL/crazyflow/actions/workflows/testing.yml
17+
18+
19+
## Architecture
20+
21+
<img src="/docs/img/architecture.png" width="75%" alt="Architecture">
22+
23+
24+
## Using the project with VSCode devcontainers
25+
26+
**Running on CPU**: by default the containers run on CPU. You don't need to take any action.
27+
28+
**Running on GPU**: The devcontsainers can easily run using your computer's NVIDIA GPU on Linux and Windows. This makes sense if you want to accelerate simulation by running thousands of simulation in parallel. In order to work you need to install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=deb_local), [NVIDIA Container runtime](https://developer.nvidia.com/container-runtime) for your computer. Finally, enable GPU access to the devcontainers by setting the commented out `"--gpus=all"` and `"--runtime=nvidia"` flags in `devcontainer.json`.
29+
30+
31+
### Linux
32+
33+
1. Install [Docker](https://docs.docker.com/engine/install/) (, and make sure Docker Daemon is running)
34+
2. Install [VSCode](https://code.visualstudio.com/), with [devcontainer extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers), and [remote dev pack](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker).
35+
3. Clone this project's code. Rename `/.devcontainer/devcontainer.linux.json` to `/.devcontainer/devcontainer.json`.
36+
4. Open this project in VSCode. VSCode should automatically detect the devcontainer and prompt you to `Reopen in container`. If not, see [here](https://code.visualstudio.com/docs/devcontainers/containers#_quick-start-open-an-existing-folder-in-a-container) to open manually. Note: Opening the container for the first time might take a while (up to 15 min), as the container is pulled from the web and build.
37+
38+
### Windows (requires Windows 10 or later)
39+
40+
For windows, we require WSL2 to run the devcontainer. (So its actually Linux with extra steps.) Full instructions can be found [in the official docs](https://code.visualstudio.com/blogs/2020/07/01/containers-wsl#_getting-started). Here are the important steps:
41+
1. Install [Docker](https://docs.docker.com/desktop/setup/install/windows-install/), and WSL2, and Ubuntu 22.04 LTS (, and make sure Docker Daemon is running)
42+
2. Docker will recognize that you have WSL installed and prompt you via Windows Notifications to enable WSL integration -> confirm this with `Enable WSL integration`. If not, open `Docker Desktop`, navigate to the settings, and manually enable WSL integration. (There are TWO setting options for this. Make sure to enable BOTH!)
43+
3. Install [VSCode](https://code.visualstudio.com/), with the [WSL extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl), [devcontainer extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers), and [remote dev pack](https://marketplace.visualstudio.com/items?itemName=ms-azuretools.vscode-docker).
44+
4. Clone the source code for the exercises in the WSL2 file system to `/home` (`~`), or wherever you like. (Performance when working on the WSL file system is much better compared to Windows file system). You can access the WSL filesystem either by starting a WSL/Ubuntu console, or via the Windows File Explorer at `\\wsl.localhost\Ubuntu\home` (replace `Ubuntu` with your distro, if necessary).
45+
7. Rename `/.devcontainer/devcontainer.windows.json` to `/.devcontainer/devcontainer.json`.
46+
8. Open this project in VSCode. The easiest way to do so is by starting a WSL/Ubuntu shell, navigating via `cd` to the source code, then type `code .` to open VSCode. VSCode should automatically detect the devcontainer and prompt you to `Reopen in container`. If not, see [here](https://code.visualstudio.com/docs/devcontainers/containers#_quick-start-open-an-existing-folder-in-a-container) to open manually. Note: Opening the container for the first time might take a while (up to 15 min), as the container is pulled from the web and build.
47+
48+
49+
### MacOS
50+
51+
Unfortunately, we did not get the devcontainer to work with MacOS yet, even after following [those](https://gist.github.com/sorny/969fe55d85c9b0035b0109a31cbcb088) steps. We expect that the issue is related to Mujoco rendering from inside the Docker container and display forwarding with X11. There is also an [unresolved Issue](https://github.com/google-deepmind/mujoco/issues/1047) on GitHub. If you manage to make it work, please let us know.
52+
53+
Until then, MacOS users are required to install this project using an python environment manager such as [conda](https://docs.anaconda.com/anaconda/install/) or [mamba](https://mamba.readthedocs.io/en/latest/). If you use conda, these are the required commands: ```conda create --name crazyflow -c conda-forge python=3.11```, ```conda activate crazyflow```, ```conda install pip```, ```pip install -e .```.
54+
55+
____________
56+
57+
Known Issues:
58+
- if building docker container fails at `RUN apt-get update`, make sure your host systems time is set correct: https://askubuntu.com/questions/1511514/docker-build-fails-at-run-apt-update-error-failed-to-solve-process-bin-sh
59+
60+
## Known Issues
61+
- `"RuntimeError: MUJOCO_PATH environment variable is not set"` upon installing this package. This error can be resolved by using `venv` instead of `conda`. Somtimes the `mujoco` install can [fail with `conda`](https://github.com/google-deepmind/mujoco/issues/1004).
62+
- If using `zsh` don't forget to escape brackets when installing additional dependencies: `pip install .\[gpu\]`.

0 commit comments

Comments
 (0)