ScaMo

Pytorch implementation of paper "ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Model"

[Project Page] [Paper]

Release Log

• 2024/12/06 Initialize the project and release the inference code.

Table of Content

• 1. Installation
• 2. Quick Start
• 3. Transform the output vector to bvh
• 4. Contribute
• 5. Acknowledgement

1. Installation

1.1. Environment

conda env create -f environment.yml
conda activate ScaMo

The code was tested on Python 3.8 and PyTorch 2.0.0.

2.2. Dependencies

bash prepare/download_model.sh

Download google flan-t5-xl model from huggingface.

bash prepare/download_t5.sh

3. Quick Start

A quick start guide of how to inference ScaMo.

python inference_generation_hf.py --nb-code 65536 --quantizer FSQ --pretrained_llama 3B --text_encode flan-t5-xl

4. Transform the output vector to bvh

You should change the folder path correctly. Here is an example:

python3 visualization/joints2bvh.py

Note the process may lead to wrong bone rotation, due to the ik. We are training a new model with a rotation-based representation to solve this problem.

5. Contribute

We try to scale the dataset and model. However, we still do not observe the emerging abilities and have a long way to go. We believe our model can "eat" more data. If you have any open-sourced data, feel free to contact me. I can contribute to convert the motion data to same format in this paper to train a better model.

6. Acknowledgement

We appreciate helps from :

• public code like t2m-gpt, momask, text-to-motion, TM2T, MDM, MotionDiffuse etc.