Difficulty in conditioning on custom input motion

[enthusiastic2003]

Hi Setareh!

I'm currently using ml-comotion to generate SMPL sequences, which I aim to convert into the HumanML3D format for use with downstream models trained on HumanML3D.

However, I'm running into an issue: the poses in HumanML3D format don't appear to match their original SMPL counterparts, even after going through the standard rescaling, alignment, and retargeting steps (e.g., uniform skeleton scaling, rotation normalization, root position adjustment, etc.).

Have there been any known issues or recommendations when converting SMPL to HumanML3D format, especially with respect to joint definitions, coordinate systems, or motion fidelity?
Any advice or references to ensure the conversion is accurate would be really appreciated.

Thanks in advance for your time and work on this project!

Best,
Sirjan.

The left one shows the SMPL Processed joints in 3D space, the right one shows the pose in HumanML3D format when fed as an input after conversion to that format suing the given scripts.

[setarehc]

Hi @enthusiastic2003,

Thansk for your interest in our work and your detailed report.

I'm not sure if you've resolved this issue already but I personally haven't worked directly on the SMPL to HumanML3D pipeline. Due to the result you've shared, there seems to be a subtle difference between the official HumanML3D preprocessing scripts and what you have used here.

Do share your insights if you've found a bug or working solution on this.

Best,
Setareh