Replaced deprecated data types and fixed some typos in README.md

.gitignore

@@ -131,6 +131,7 @@ dmypy.json
 .pyre/
 
 # experiments
+working/
 save/
 dataset/
 kit/
@@ -139,3 +140,19 @@ glove/
 body_models/
 .vscode/
 wandb/
+
+# vim swaps
+.*.sw?
+
+# tarballs and zips
+*.tar
+*.tar.gz
+*.tgz
+*.zip
+*.gz
+*.7z
+*.bzip
+*.bz2
+
+# numpy arrays
+*.np[yz]

README.md

@@ -29,6 +29,7 @@ If you find this code useful in your research, please cite:
 ## Getting started
 
 This code was developed on `Ubuntu 20.04 LTS` with Python 3.7, CUDA 11.7 and PyTorch 1.13.1.
+The current `requirements.txt` was set up with Python 3.9, CUDA 11.3, PyTorch 1.12.1.
 
 
 ### 1. Setup environment
@@ -46,12 +47,10 @@ This codebase shares a large part of its base dependencies with [GMD](https://gi
 
 Setup virtual env:
 ```shell
-python3 -m venv .env_condmdi
-source .env_condmdi/bin/activate
-pip uninstall ffmpeg
-pip install spacy
-python -m spacy download en_core_web_sm
-pip install git+https://github.com/openai/CLIP.git
+python3 -m venv .env_condmdi      # pick your preferred name here
+source .env_condmdi/bin/activate  # and use that name in place of .env_condmdi
+pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
+pip install -r requirements.txt   # updated to include spacy and clip configuration
 ```
 
 Download dependencies:
@@ -78,36 +77,40 @@ bash prepare/download_recognition_unconstrained_models.sh
 ### 2. Get data
 There are two paths to get the data:
 
-(a) **Generation only** wtih pretrained text-to-motion model without training or evaluating
-
-(b) **Get full data** to train and evaluate the model.
-
+<details>
+    <summary><b>(a) **Generation only** with pretrained text-to-motion model without training or evaluating</b></summary>
 
-#### a. Generation only (text only)
+    #### a. Generation only (text only)
 
-**HumanML3D** - Clone HumanML3D, then copy the data dir to our repository:
+    **HumanML3D** - Clone HumanML3D, then copy the data dir to our repository:
 
-```shell
-cd ..
-git clone https://github.com/EricGuo5513/HumanML3D.git
-unzip ./HumanML3D/HumanML3D/texts.zip -d ./HumanML3D/HumanML3D/
-cp -r HumanML3D/HumanML3D diffusion-motion-inbetweening/dataset/HumanML3D
-cd diffusion-motion-inbetweening
-cp -a dataset/HumanML3D_abs/. dataset/HumanML3D/
-```
+    ```shell
+    cd ..
+    git clone https://github.com/EricGuo5513/HumanML3D.git
+    unzip ./HumanML3D/HumanML3D/texts.zip -d ./HumanML3D/HumanML3D/
+    cp -r HumanML3D/HumanML3D diffusion-motion-inbetweening/dataset/HumanML3D
+    cd CondMDI
+    cp -a dataset/HumanML3D_abs/. dataset/HumanML3D/
+    ```
+</details>
 
+<details>
+    <summary><b>(b) **Get full data** to train and evaluate the model.</b></summary>
 
-#### b. Full data (text + motion capture)
+    #### b. Full data (text + motion capture)
 
-**[Important !]**
-Following GMD, the representation of the root joint has been changed from relative to absolute. Therefore, you need to replace the original files and run GMD's version of `motion_representation.ipynb` and `cal_mean_variance.ipynb` provided in `./HumanML3D_abs/` instead to get the absolute-root data.
+    **HumanML3D** - Follow the instructions in [HumanML3D](https://github.com/EricGuo5513/HumanML3D.git),
+    then copy the result dataset to our repository:
 
-**HumanML3D** - Follow the instructions in [HumanML3D](https://github.com/EricGuo5513/HumanML3D.git),
-then copy the result dataset to our repository:
+    **[Important !]**
+    Following GMD, the representation of the root joint has been changed from relative to absolute. Therefore, when setting up HumanML3D, please
+    run GMD's version of `motion_representation.ipynb` and `cal_mean_variance.ipynb` instead to get the absolute-root data. These files are made
+    available in `./dataset/HumanML3D_abs/`.
 
-```shell
-cp -r ../HumanML3D/HumanML3D ./dataset/HumanML3D
-```
+    ```shell
+    cp -r ../HumanML3D/HumanML3D ./dataset/HumanML3D
+    ```
+</details>
 
 ### 3. Download the pretrained models
 
@@ -156,7 +159,7 @@ python -m sample.conditional_synthesis --model_path ./save/condmdi_randomframes/
   <summary><b>Text to Motion - <u>With</u> keyframe conditioning</b></summary>
 
 ### Generate from a single prompt - condition on keyframe locations
-#### using the uncoditioned model
+#### using the unconditioned model
 ```shell
 python -m sample.edit --model_path ./save/condmdi_uncond/model000500000.pt --edit_mode benchmark_sparse --transition_length 5 --num_samples 10 --num_repetitions 3 --imputate --stop_imputation_at 1 --reconstruction_guidance --reconstruction_weight 20 --text_condition "a person throws a ball"
 ```
@@ -189,7 +192,7 @@ python -m sample.conditional_synthesis --model_path ./save/condmdi_randomframes/
 * `--device` id.
 * `--seed` to sample different prompts.
 * `--motion_length` (text-to-motion only) in seconds (maximum is 9.8[sec]).
-* `--progress` to save the denosing progress.
+* `--progress` to save the denoising progress.
 
 **Running those will get you:**
 * `results.npy` file with text prompts and xyz positions of the generated animation
@@ -227,11 +230,11 @@ Our model is trained on the **HumanML3D** dataset.
 ```shell
 python -m train.train_condmdi --keyframe_conditioned
 ```
-* You can ramove `--keyframe_conditioned` to train a unconditioned model.
+* You can remove `--keyframe_conditioned` to train a unconditioned model.
 * Use `--device` to define GPU id.
 
 ## Evaluate
-All evaluation are done on the HumanML3D dataset.
+All evaluations are done on the HumanML3D dataset.
 
 ### Text to Motion - <u>With</u> keyframe conditioning
 
@@ -247,7 +250,7 @@ python -m eval.eval_humanml_condmdi --model_path ./save/condmdi_uncond/model0005
 
 #### on the conditional model
 ```shell
-python -m eval.eval_humanml_condmdi --model_path ./save/condmdi_randomframes/model000750000.pt --edit_mode gmd_keyframes --keyframe_guidance_param 1.
+python -m eval.eval_humanml_condmdi --model_path ./save/condmdi_randomframes/model000750000.pt --edit_mode gmd_keyframes --keyframe_guidance_param 1
 ```
 
 
@@ -260,4 +263,4 @@ We would like to thank the following contributors for the great foundation that
 ## License
 This code is distributed under an [MIT LICENSE](LICENSE).
 
-Note that our code depends on other libraries, including CLIP, SMPL, SMPL-X, PyTorch3D, and uses datasets that each have their own respective licenses that must also be followed.
+Note that our code depends on other libraries, including CLIP, SMPL, SMPL-X, PyTorch3D, and uses datasets that each have their own respective licenses that must also be followed.

configs/card.py

@@ -75,6 +75,12 @@ class motion_abs_unet_adagn_xl(
 ):
     save_dir: str = 'save/unet_adazero_xl_x0_abs_loss1_fp16_clipwd_224'
 
+@dataclass
+class motion_abs_unet_adagn_xl_custom_batch(
+    data.humanml_motion_abs,
+    model.motion_unet_adagn_xl,
+):
+    batch_size: int = 2         ## change the batch size here
 
 @dataclass
 class motion_abs_unet_adagn_xl_loss2(

data_loaders/amass_utils.py

@@ -4,35 +4,35 @@
 
 
 # Matrix that shows joint correspondces to SMPL features
-MAT_POS = np.zeros((24, 764), dtype=np.bool)
+MAT_POS = np.zeros((24, 764), dtype=bool)
 MAT_POS[0, :3] = True # root position = trans
 for joint_idx in range(24):
     ub = 3 + 24*3*3 + 3 * (joint_idx + 1)
     lb = ub - 3
     MAT_POS[joint_idx, lb:ub] = True # joint position = pos
 
-MAT_ROTMAT = np.zeros((24, 764), dtype=np.bool) # rotmat = 24,3,3 wrp to the parent joint
+MAT_ROTMAT = np.zeros((24, 764), dtype=bool) # rotmat = 24,3,3 wrp to the parent joint
 for joint_idx in range(24):
     ub = 3 + 3*3 * (joint_idx + 1)
     lb = ub - 9
     MAT_ROTMAT[joint_idx, lb:ub] = True # joint rotation = rotmat
 
-MAT_HEIGHT = np.zeros((24, 764), dtype=np.bool) # height = 24
+MAT_HEIGHT = np.zeros((24, 764), dtype=bool) # height = 24
 for joint_idx in range(24):
     ub = 3 + 24*3*3 + 24*3 + 24*3 + 8 + (joint_idx + 1)
     lb = ub - 1
     MAT_HEIGHT[joint_idx, lb:ub] = True # joint rotation = rotmat
 
-MAT_ROT6D = np.zeros((24, 764), dtype=np.bool) # rot2d = 24,2 wrp to the parent joint
+MAT_ROT6D = np.zeros((24, 764), dtype=bool) # rot2d = 24,2 wrp to the parent joint
 for joint_idx in range(24):
     ub = 3 + 24*3*3 + 24*3 + 24*3 + 8 + 24 + 3 + 24*3 + 24*6 + 6 + 6 * (joint_idx + 1)
     lb = ub - 6
     MAT_ROT6D[joint_idx, lb:ub] = True # joint rotation = rotmat
 
-MAT_ROT = np.zeros((24, 764), dtype=np.bool) # global_xform = 24, 6 wrp to the root
+MAT_ROT = np.zeros((24, 764), dtype=bool) # global_xform = 24, 6 wrp to the root
 lb = 3 + 24*3*3 + 24*3 + 24*3 + 8 + 24 + 3 + 24*3 + 24*6
 MAT_ROT[0, lb:lb+6] = True # root rotation = root_orient
 for joint_idx in range(24):
     ub = 3 + 24*3*3 + 24*3 + 24*3 + 8 + 24 + 3 + 24*3 + (joint_idx + 1) * 6
     lb = ub - 6
-    MAT_ROT[joint_idx, lb:ub] = True # joint rotation = global_xform
+    MAT_ROT[joint_idx, lb:ub] = True # joint rotation = global_xform