Describe the bug
When using vab-omnigibson for evaluation for Qwen2VL-72B-Instruct and Qwen2.5VL-72B-Instruct, we find that models consistently make failures when performing "put_inside" function, which leads models to fail most of the scenes. It doesn't seem like the model's fault, as in tasks like "dispose_of_a_pizza_box" put inside often happen in the third round (after grasp and move).
Screenshots or Terminal Copy&Paste
An example of failure in "dispose_of_a_pizza_box"
The visual input of Round 3
Describe the bug
When using vab-omnigibson for evaluation for Qwen2VL-72B-Instruct and Qwen2.5VL-72B-Instruct, we find that models consistently make failures when performing "put_inside" function, which leads models to fail most of the scenes. It doesn't seem like the model's fault, as in tasks like "dispose_of_a_pizza_box" put inside often happen in the third round (after grasp and move).
Screenshots or Terminal Copy&Paste
An example of failure in "dispose_of_a_pizza_box"
The visual input of Round 3
