Successfully ran sim-to-sim Mujoco Evaluation in `TextOpTracker`, but ONNX Controller test without RobotMDAR in `TextOpDeploy` fails

[3b1b-sh]

Environment

• OS: Ubuntu 22.04
• ROS2: Humble

---

1. TextOpTracker: Mujoco evaluation works as expected

I can successfully run the Mujoco evaluation in TextOpTracker:

(env_isaaclab) ehzoah@ehzoah-desktop:~/textop/textop/TextOpTracker$ python scripts/deploy_mujoco.py \
  --motion_path=/home/ehzoah/textop/textop/TextOpTracker/artifacts/shoulder_motions/50002_shake_shoulders_stageii/motion.npz \
  --policy_path=/home/ehzoah/textop/textop/TextOpTracker/logs/rsl_rl/Pretrained/checkpoints/exported/policy.onnx

The script runs without errors and prints key parameters such as kp, kd, action_scale, default joint positions, and joint reindex mappings.
The Mujoco simulation completes all 105 timesteps successfully.

Example logs:

kp:  [ 200. 200. ... 200.]
kd:  [10. 10. ... 10.]
action_scale: 0.5
default_angles_: 0 0 -0.4 0.8 -0.4 0 ...
mujoco_to_isaaclab_reindex_: 0 1 2 ... 28
isaaclab_to_mujoco_reindex_: 0 1 2 ... 28

Note: some timestep logs appear truncated due to stdout formatting, but the simulation completes all 105 steps normally.

---

2. TextOpDeploy: ONNX controller fails without RobotMDAR

Terminal 1 – Launch Mujoco simulation

(textop) ehzoah@ehzoah-desktop:~/textop/textop/TextOpDeploy/src/unitree_mujoco/simulate/build$ ./unitree_mujoco -r g1 -n enp0s31f6 -i 0

The simulation initializes successfully with all links, joints, actuators, and sensors (29 DOF).

---

Terminal 2 – Launch ONNX controller

(textop) ehzoah@ehzoah-desktop:~/textop/textop/TextOpDeploy$ ros2 launch textop_ctrl textop_onnx_controller.launch.py \
  onnx_model:=/home/ehzoah/textop/textop/TextOpTracker/logs/rsl_rl/Pretrained/checkpoints/exported/policy.onnx \
  motion_file:=/home/ehzoah/textop/textop/TextOpTracker/artifacts/shoulder_motions/50002_shake_shoulders_stageii/motion.npz \
  config_file:=/home/ehzoah/textop/textop/TextOpDeploy/install/textop_ctrl/share/textop_ctrl/config/g1_29dof.yaml

The controller initializes successfully:

• ONNX model loaded
• Motion file loaded
• Observation dimension: [1, 431]
• Action dimension: [1, 29]
• Default joint angles and reindex mappings printed

However, the controller immediately exits due to a delta-q safety check:

[textop_onnx_controller-1] [ERROR]: delta q 0 is too large.
[textop_onnx_controller-1] target q    : 3.344359
[textop_onnx_controller-1] target dq   : 0.000000
[textop_onnx_controller-1] q           : -0.348966
[textop_onnx_controller-1] dq          : -7.031245
[textop_onnx_controller-1] [ERROR]: Safe exit triggered - shutting down controller

---

Terminal 3 – Publish motion

(py310) ehzoah@ehzoah-desktop:~/textop/textop/TextOpDeploy$ python src/textop_ctrl/scripts/npz_motion_publisher.py --mode single /home/ehzoah/textop/textop/TextOpTracker/artifacts/shoulder_motions/50002_shake_shoulders_stageii/motion.npz

The motion is published successfully, but the controller still exits with the same delta-q error.

---

Question:

I am puzzled why the same ONNX policy and motion file work correctly in TextOpTracker Mujoco evaluation, but testing the ONNX controller in TextOpDeploy fails without RobotMDAR. Is there an extra configuration or step required for the deploy part that I might be missing?

Any guidance would be greatly appreciated.

[Shi-Soul]

your usage looks correct. some guess:

1. is the motion very conplex?
2. does the first frame of the motion significantly differ from the default stand pose?
3. can you remove the safe guard and check whether the result changes?

Provide the videos of both situations would help us understand the problem.

[clearlove-Yu]

"Hi, I'm also trying to reproduce this project. I am encountering failures when using the ONNX controller without RobotMDAR.

I have a few questions regarding the simulation:

  1. Does simulate.cc run directly, or does it need modification?
  
  2. My simulator in Terminal 2 is not rendering the robot properly.
  
  3. Terminal 1 is not receiving the low state from Terminal 2.

I'd love to discuss this with you. My email is 24B953038@stu.hit.edu.cn."

[clearlove-Yu]

I've solved my problem, and now I have the same problem as you。“[textop_onnx_controller-1] [WARN] [1766581238.444138253] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] [WARN] [1766581239.444151334] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] [WARN] [1766581240.444275126] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] [WARN] [1766581241.464176987] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] [WARN] [1766581242.464219589] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] [WARN] [1766581243.464280614] [textop_onnx_controller]: No low state received yet
[textop_onnx_controller-1] In [Zero Torque] Mode, Press 'Start' to continue
[textop_onnx_controller-1] [INFO] [1766581265.277492535] [textop_onnx_controller]: Motion sequence updated: T=350, block_size=350
[textop_onnx_controller-1] In [Damping] Mode, Press 'A' to continue
[textop_onnx_controller-1] [ERROR] [1766581274.984605067] [textop_onnx_controller]: delta q 15 is too large.
[textop_onnx_controller-1] target q : 0.000000
[textop_onnx_controller-1] target dq : 0.000000
[textop_onnx_controller-1] q : -3.089460
[textop_onnx_controller-1] dq : 0.000008
[textop_onnx_controller-1]
[textop_onnx_controller-1] [ERROR] [1766581274.984661204] [textop_onnx_controller]: Safe exit triggered - shutting down controller
[INFO] [textop_onnx_controller-1]: process has finished cleanly [pid 7531]
(textop) root@yu:/workspace/TextOp/TextOpDeploy# ”