Skip to content

README.md

Latest commit

 

History

History
125 lines (95 loc) · 4.21 KB

README.md

File metadata and controls

125 lines (95 loc) · 4.21 KB

Utility scripts

Back to main README

Tools for ClearML tasks management

Note

Be sure to get the ClearML project structure to this:

PROJECTS/.../
└── YOUR_PROJECT/
    ├── EXPERIMENT_1/
    │   ├── SUMMARY
    │   ├── trial_run_1
    │   ├── trial_run_2
    │   ├── ...
    │   └── trial_run_N
    ├── EXPERIMENT_2/
    │   ├── SUMMARY
    │   ├── trial_run_1
    │   ├── trial_run_2
    │   ├── ...
    │   └── trial_run_N
    ├── EXPERIMENT_3/
    │   ├── SUMMARY
    │   ├── trial_run_1
    │   ├── trial_run_2
    │   ├── ...
    │   └── trial_run_N
    └── EXPERIMENTS_SUMMARY

where SUMMARY is the result of the clearml_summarizer.py run in the YOUR_PROJECT/EXPERIMENT_X project name path, and the EXPERIMENTS_SUMMARY is the result of the clearml_exp_plotter.py run in the YOUR_PROJECT project name path.

Example usage

# 1. Schedule tasks to a given queue
./clearml_enqueue_tasks.py --base-task-id=BASE-TASK-ID --tasks-num=20 --queue-name=robolab

# 2. Summarize experiments
./clearml_summarizer.py --cleanup-previous-tags --project-name PROJ_EXP1/architecture_rl-sim/net64_temper_actor --tags net64 hyperopt
./clearml_summarizer.py --cleanup-previous-tags --project-name PROJ_EXP1/architecture_rl-sim/net128_temper_actor --tags net128 hyperopt
./clearml_summarizer.py --cleanup-previous-tags --project-name PROJ_EXP1/architecture_rl-sim/net256_temper_actor --tags net256 hyperopt
./clearml_summarizer.py --cleanup-previous-tags --project-name PROJ_EXP1/architecture_rl-sim/net512_temper_actor --tags net512 hyperopt

# 3. Aggregate experiment summaries
./clearml_exp_plotter.py --cleanup-previous-tags --project-name PROJ_EXP1/architecture_rl-sim -h

clearml_enqueue_tasks.py

Clone task multiple times into a selected queue.

./clearml_enqueue_tasks.py --base-task-id=BASE-TASK-ID --tasks-num=20 --queue-name=geonosis

clearml_enqueue_eval_tasks.py

For each completed training task in a source project, clone an eval template and enqueue.

./clearml_enqueue_eval_tasks.py --eval-template-id=TEMPLATE-TASK-ID --source-project=PROJECT-PATH --queue-name=geonosis

clearml_exp_plotter.py

Aggregate summaries of multiple ClearML experiments into comparison plots!

Usage

To use it properly, you need to run the clearml_summarizer.py first.

uv run --script ./clearml_exp_plotter.py --cleanup-previous-tags --project-name PROJECT/PATH
uv run --script ./clearml_exp_plotter.py -h

clearml_summarizer.py

Summarize ClearML experiments.

Usage

uv run --script ./clearml_summarizer.py --cleanup-previous-tags --project-name PROJECT/PATH --tags bc
uv run --script ./clearml_summarizer.py -h

Dataset manipulation

upload_urdf_to_clearml.py

Upload URDF model, required by simulators, into ClearML.

Uploading

  1. Generate standalone URDF model with aegis_ros/aegis_description launch command:
ros2 launch aegis_description generate_standalone_urdf.launch.py disable_cell:=true

Which will generate the whole URDF file with 3D models in a default ~/ceai_ws/aegis_urdf directory.

  1. Run the utils/upload_urdf_to_clearml.py script with the following options:
python3 utils/upload_urdf_to_clearml.py ~/ceai_ws/aegis_urdf --name AegisURDFModel --project AEGIS_GRASP --desc "Aegis simulator assets"

Warning

To update the dataset make sure to add an additional option: --parent "PREVIOUS_DATASET_ID"

  1. Check the ClearML server's datasets.

Usage

In the robot's config set the urdf_model_id param to the ClearML's dataset ID.

Important

In case of failure to obtain the model, the code will try to load URDF model from ~/ceai_ws/aegis_urdf directory.

Back to main README