Skip to content

Fix/replace slices with string #2571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

LinghengMeng
Copy link
Contributor

Description

This request is to fix the problem of running Isaac-Stack-Cube-Franka-v0 with rsl_rl PPO.

The changes are around the Isaac-Stack-Cube-Franka-v0:

  • replace_slices_with_strings() and replace_strings_with_slices() in IsaacLab/source/isaaclab/isaaclab/utils/dict.py are changed to allow slices being process if a list of dicts is passed in as input.
  • add rsl_rl_ppo_cfg.py to Isaac-Stack-Cube-Franka-v0 and update gym.register
  • set concatenate_terms to True in IsaacLab/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/stack_env_cfg.py. Otherwise, RL lib following Gymnasium API will not work when setting up observation dimension with obs_dim = obs.shape[1]

To test, run the following:

cd IsaacLab
./isaaclab.sh -p ./scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Stack-Cube-Franka-v0  --num_envs 10

Note: Because there is no RewardCfg for this task, there will be no learning progress. This can only check the task can be run without any problem.

Fixes #2481

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Before After
isaaclab_before isaaclab_after
  • Change 2: add rsl_rl_ppo_cfg.py to Isaac-Stack-Cube-Franka-v0 and update gym.register
Before After
change2_add_rsl_rl_ppo_cfg
change2_before change2_after
  • Change 3: set concatenate_terms to True in IsaacLab/source/isaaclab_tasks/isaaclab_tasks/manager_based/manipulation/stack/stack_env_cfg.py. Otherwise, RL lib following Gymnasium API will not work when setting up observation dimension with obs_dim = obs.shape[1]
Before After
change3_before change3_after

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@@ -29,6 +29,7 @@
entry_point="isaaclab.envs:ManagerBasedRLEnv",
kwargs={
"env_cfg_entry_point": stack_joint_pos_env_cfg.FrankaCubeStackEnvCfg,
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:StackCubePPORunnerCfg",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if we want to add a RL agent cfg for the task, would be good to also extend the task to allow for learning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify what this task is used for? Do we only want to run it with a random agent?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

current use case for it is only for imitation learning, so it does not have some of the components like rewards needed for reinforcement learning

@@ -267,6 +267,8 @@ def replace_slices_with_strings(data: dict) -> dict:
"""
if isinstance(data, dict):
return {k: replace_slices_with_strings(v) for k, v in data.items()}
elif isinstance(data, list):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might make sense to separate these changes apart from the environment updates so that we can merge these in individually.

kellyguo11 added a commit that referenced this pull request Jul 7, 2025
# Description

replace_slices_with_strings() and replace_strings_with_slices() in
[IsaacLab/source/isaaclab/isaaclab/utils/dict.py](https://github.com/isaac-sim/IsaacLab/blob/main/source/isaaclab/isaaclab/utils/dict.py)
are changed to allow slices being process if a list of dicts is passed
in as input.

Cherry picks changes from
#2571 by @LinghengMeng

Fixes #2481 


## Type of change

<!-- As you go through the list, delete the ones that are not
applicable. -->

- Bug fix (non-breaking change which fixes an issue)

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [ ] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

<!--
As you go through the checklist above, you can mark something as done by
putting an x character in it

For example,
- [x] I have done this task
- [ ] I have not done this task
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug Report] omegaconf.errors.UnsupportedValueType Error
2 participants