Home

LeRobot Setup

Motor control driver board: ESP32 Adapter 12V

- Connect to the SSID: ESP32_DEV
- Password: 12345678

manual : https://www.waveshare.com/w/upload/f/f4/ST3215_Servo_User_Manual.pdf#page=4.43

After logging in set the mode to Servo Mode and Start Serial Forwarding

Next run configure_motor script in lerobot repo.

python3 lerobot/scripts/configure_motor.py --port /dev/ttyUSB0 --brand feetech --model sts3215 --baudrate 115200 --ID 6

Robot Config:

You can configure the robot or change the existing config at

lerobot/lerobot/common/robot_devices/robots/configs.py

Streaming the USB cam or Laptop cam on Server for Inference

Laptop/Desktop:

• Download go2rtc from https://github.com/AlexxIT/go2rtc/releases/tag/v1.9.9
• Run the gp2rtc app from the terminal
• Open webUI http://localhost:1984/
• Go to config and add your config (find streamable device from the "add" tab on http://localhost:1984/add.html)

streams:
  usb_cam:
    - v4l2:device?video=/dev/video2&input_format=mjpeg&video_size=1280x800&framerate=30

Server:

Add yourself to the video group

sudo usermod –a –G video $USER

On the server reload v4l2loopback kernel module and add your virtual device port

prai01@prai01:~$ sudo modprobe -r v4l2loopback
prai01@prai01:~$ sudo modprobe v4l2loopback devices=1 video_nr=2 exclusive_caps=1 card_label="Virtual Webcam"
prai01@prai01:~$ sudo v4l2-ctl --list-devices
Virtual Webcam (platform:v4l2loopback-000):
    /dev/video2

Find your IP and stream the video to the server virtual port

prai01@prai01:~$ sudo ffmpeg -i rtsp://192.168.3.69:8554/usb_cam -f v4l2 -vcodec rawvideo -pix_fmt yuv420p /dev/video2 

This should start streaming your data to the virtual port and ready to be used in your lerobot code

pip3 install -U hello-robot-stretch-body-tools
pip3 install -U hello-robot-stretch-body

Attach USB to the remote server

Local Machine (find the appropriate usbid number corresponding to your device using dmesg)

sudo modprobe usbip-core
sudo modprobe usbip-host
sudo usbip bind -b 1-11

Start the usbip server

sudo usbipd -D

Remote Machine

sudo modprobe vhci-hcd
sudo usbip attach -r <local_machine_ip> -b 1-11