About KIT dataset

[JosephKKim]

Hello, @EricGuo5513 thanks for releasing this dataset!

While processing dataset with the given raw_pose_processing.ipynb file, I got an error that says, no file named ./pose_data/KIT/3/kick_high_left02_poses.npy,
so I looked into the KIT dataset which I've downloaded from the AMASS webpage,
( I've downloaded SMPL-X N from the AMASS webpage, which gives me KIT.tar.bz2)
the file extension was .npz instead of .npy.

I tried to change its name to .npz but it gives the error message which says
KeyError: 'slice(0, 117, None) is not a file in the archive'

In this case, what would be the problem?
thanks!

Joseph

[EricGuo5513]

Hi, What is the shape of this array?

…

On Mon, Jan 2, 2023 at 4:16 AM JosephKKim ***@***.***> wrote: Hello, @EricGuo5513 <https://github.com/EricGuo5513> thanks for releasing this dataset! While processing dataset with the given raw_pose_processing.ipynb file, I got an error that says, no file named ./pose_data/KIT/3/kick_high_left02_poses.npy, so I looked into the KIT dataset which I've downloaded from the AMASS webpage, the file extension was .npz instead of .npy. I tried to change its name to .npz but it gives the error message which says KeyError: 'slice(0, 117, None) is not a file in the archive' In this case, what would be the problem? thanks! Joseph — Reply to this email directly, view it on GitHub <#25>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKRYNB3VVXPUYEMEVL5MTYLWQK2IHANCNFSM6AAAAAATOXJLLM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JosephKKim]

It seems like after np.load .npz files, the variable named data is still a npz file.

These are my KIT dataset, as you can see all files have npz extensions.

for more information this comes out from the ipynb file

[JosephKKim]

In addition, I just unrar dataset given in this repository (new_joint_vecs and new_joints) but it gives error while running cal_mean_variance.ipynb

The error is as the screen capture below.

[EricGuo5513]

Hello, sorry for the late reply. You need to download SMPL+H G on AMASS website, instead of SMPL-X N. Not sure if that triggers the error.

The raw_pose_process script should be able to load with whatever data formate AMASS provides, as the SMPL part follows https://github.com/nghorbani/amass/tree/master/notebooks. In this case, the script should load npz data rather than npy data. We just save the data in npy format.

For cal_mean_variance script, HumanML3D dataset should have dimension of 263. Could you make sure that you have passed 22 to argument joints_num in mean_variance.

[JosephKKim]

No it's okay, I know that every researchers are living very busy life👍🏻

So let me wrap up this issue real quick.

1. If I want to process from the raw AMASS dataset I should download SMPL + H G
2. If I don't want to start from the raw data, I should unzip the rar files you provided and unrar them -> get new_joint_vecs and use them to run rest of the .ipynb files
   right?

But, when I tried to validate those files through running cal_mean_variance I'm getting error like the one below.

What should I do?

[EricGuo5513]

Hi, I got it. I think what you downloaded is KIT-ML dataset, it’s not or not part of HumanML3D. It has different skeletons and number of joints. To obtain HumanML3D, you have to download AMASS and run the scripts from scratch, as we can not spread AMASS publicly. Best, Chuan

…

On Mon, Jan 9, 2023 at 6:43 PM JosephKKim ***@***.***> wrote: No it's okay, I know that every researchers are living very busy life👍🏻 So let me wrap up this issue real quick. 1. If I want to process from the raw AMASS dataset I should download SMPL + H G 2. If I don't want to start from the raw data, I should unzip the rar files you provided and unrar them -> get new_joint_vecs and use them to run rest of the .ipynb files right? But, when I tried to validate those files through running cal_mean_variance I'm getting error like the one below. [image: image] <https://user-images.githubusercontent.com/56065530/211442733-80cdb5e2-8cd2-4106-a708-400bea71c569.png> What should I do? — Reply to this email directly, view it on GitHub <#25 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKRYNB5HZKNAW5FJNZCNIDLWRS5EZANCNFSM6AAAAAATOXJLLM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[EricGuo5513]

No it's okay, I know that every researchers are living very busy life👍🏻

So let me wrap up this issue real quick.

1. If I want to process from the raw AMASS dataset I should download SMPL + H G
2. If I don't want to start from the raw data, I should unzip the rar files you provided and unrar them -> get new_joint_vecs and use them to run rest of the .ipynb files
   right?

But, when I tried to validate those files through running cal_mean_variance I'm getting error like the one below.

What should I do?

Where did you get these new_joint_vecs

[JosephKKim]

Just by unrar new_joint_vecs.rar from your Google Drive.

[EricGuo5513]

Just by unrar new_joint_vecs.rar from your Google Drive.

Could you send me the link? It seems the data you obtained is KIT-ML dataset, instead of HumanML3D.

[HuYicheng]

Hello, @EricGuo5513 . I just encounter the same problem. I have download KIT SMPL + H G. And I try to run this script raw_pose_processing.ipynb

Then failed with this warning

FileNotFoundError: [Errno 2] No such file or directory: './pose_data/KIT/3/kick_high_left02_poses.npy'

In this directory, the file extention is npz rather than npy.

[JosephKKim]

Sorry for the late reply,
So, since AMASS dataset is the collection of human motion capture dataset, I have downloaded all data from the official site, AMASS.
After that, following the README from the repository I have downloaded KIT-ML dataset from KIT-ML, and I think new_joint_vecs.rar and new_joints.rar are KIT-ML dataset(Am I right?).

So from the discussion before, what KIT-ML and HumanML3D is different dataset, and let's focus on the HumanML3D.
I think I followed all the instruction from the raw_pose_processing.ipynb, still getting the same error message @HuYicheng is pointing out. I think users will need more specific and detailed instruction in order to get HumanML dataset correctly. I know that making README instructions is very time consuming work, but it would help more confused users like me.

Thanks.

[HuYicheng]

Hi@JosephKKim. Have you resolved the problem and contructed HumanML3D dataset successfully after downloading the whole data from AMASS?

[JosephKKim]

@HuYicheng
No, I'm still working on it and found something, while looking on the raw_pose_processing file, I think the problem is with the 4th cell.
printing the variable named dataset_names, which is a list. It gives an empty list.

If this is the case I think we should fix the code.
Tell me if you are getting the same result.

[JosephKKim]

Hi, @HuYicheng
I found the solution for making .npy file, and also what was missing in the .ipynb instruction which caused the confusion in our communication between the author and users.

First, after reading the text part in the given notebook file, I found out that the authors is recommending to process 1 sub-dataset at the time (as you know that AMASS is consist of many sub-datasets which I refer to sub-dataset is something like KIT, SFU, MPI_Limits ... etc.).

So in the 4th line of the 4th cell, which is the code snippet below,
for root, dirs, files in os.walk('./amass_data'):
you have to add the sub-dataset to the path inside the os.walk. For example, for the KIT sub-dataset
os.walk('./amass_data/KIT')

After that you can see that it is processing with the raw data which is in npz formats and converting it into npy formats.

Hope that it would help you and other users who might be confused about using this HumanML3D dataset.

DONE!