Skip to content

BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO

License

Notifications You must be signed in to change notification settings

BricksRL/bricksrl

Repository files navigation

BricksRL

CI Python arXiv Website

BricksRL allows the training of custom LEGO robots using deep reinforcement learning. By integrating PyBricks and TorchRL, it facilitates efficient real-world training via Bluetooth communication between LEGO hubs and a local computing device. Check out our paper!

For additional information and building instructions for the robots, view the project page BricksRL.

Prerequisites

Click me

Enable web Bluetooth on chrome

  1. Go to "chrome://flags/"
  2. enable "Experimental Web Platform features"
  3. restart chrome
  4. Use beta.pybricks.com to edit and upload the client scripts for each environment

Environment Setup

  1. Create a Conda environment:
    conda create --name bricksrl python=3.8
  2. Activate the environment:
    conda activate bricksrl
  3. Install PyTorch:
    pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
    
  4. Install additional packages:
    pip install -r requirements.txt

Usage

Client

Update your client script on the PyBricks Hub whenever you want to run a new environment with your robot.

Repo Structure

Click me
project_root/
│
├── configs/                    # Centralized configuration directory
│   ├── config.yaml             # Base config
│   ├── env/                    # Environment and task specific configs
|   |   ├── runaway-v0.yaml
|   |   ├── spinning_v0.yaml
|   |   ├── walker-v0.yaml
|   |   ├── walker_sim-v0.yaml
|   |   ├── roboarm-v0.yaml
|   |   ├── roboarm_sim-v0.yaml
|   |   └── roboarm_mixed-v0.yaml  
│   └── agent/                   # Agent specific configs
|       ├── sac.yaml
|       ├── td3.yaml
|       └── droq.yaml
│   
├── experiments/                # Experiments directory
│   ├── 2wheeler/               # 2wheeler robot specific experiments
|   |   ├── train.py
|   |   └── eval.py  
│   ├── walker/                 # Walker robot specific experiments
|   |   ├── train.py
|   |   └── eval.py  
│   └── roboarm/                # Roboarm specific experiments
|       ├── train.py
|       └── eval.py  
│
├── environments/               # Environments directory
│   ├── __init__.py
│   ├── base/                   # Base environment class
|   |   ├── base_env.py
|   |   └── PybricksHubClass.py # For Async-Communication with the robot
│   ├── runaway_v0.py           # Environment for the 2wheeler robot
|   |   ├── client.py
|   |   └── Env.py  
│   ├── walker_v0.py            # Environment for the walker
|   |   ├── client.py
|   |   └── Env.py
│   └── ...
│
├── src/                     # Source code for common utilities, robot models, etc.
│   ├── __init__.py
│   ├── utils/
│   ├── agents/
|   |   ├── sac.py
|   |   └── td3.py
│   └── networks/
|       └── ...
│
└── tests/                   # Unit tests and integration tests
    ├── ...

Config

Before running experiments, please review and modify the configuration settings according to your needs. Each environment and agent setup has its own specific configuration file under the configs/ directory. For more information checkout the config README.

Robots

Robots utilized for our experiments. Building instructions can be found here.

2wheeler Walker RoboArm
2Wheeler Walker RoboArm

Run Experiments

Train an Agent

python experiments/walker/train.py

Evaluate an Agent

python experiments/walker/eval.py

Results

Click me

Evaluation videos of the trained agents can be found here.

2Wheeler Results:

2Wheeler Results

Walker Results:

Walker Results

RoboArm Results:

RoboArm Results RoboArm Mixed Results

About

BricksRL: A Platform for Democratizing Robotics and Reinforcement Learning Research and Education with LEGO

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages