C3D Finetuning

[sakerlrf]

Hello,

Following the steps here, I am attempting finetuning c3d_sports1m_pretrained.py, for a new dataset of 2 classes.

If not mistaken, there are 3 files related:

• i3d_head.py being set in--> c3d_sports1m_pretrained.py being set in--> c3d_sports1m_16x1x1_45e_ucf101_rgb.py

To train only the head (update only the head), and keep the c3d_sports1m_pretrained.py weights as they are, should I use:

• load_from = c3d_sports1m_pretrain_20201016-dcc47ddc.pth, in the c3d_sports1m_16x1x1_45e_ucf101_rgb.py or not?

and, is there any difference between:

• _base_ = '../../_base_/models/c3d_sports1m_pretrained.py', and
• load_from = c3d_sports1m_pretrain_20201016-dcc47ddc.pth, in the c3d_sports1m_16x1x1_45e_ucf101_rgb.py?

Thanks a lot.

[dreamerlin]

Since only the difference is head in your case, just change the load_from and num_classes.

[dreamerlin]

If you find our repo helpful, hope you can recommend it to your friends : )

[sakerlrf]

@dreamerlin, thanks a lot for the guidance!

It should come with no doubt that one would definitely share the experience of using MM🌟CTION2!!
Who could get bored exploring it?!

Clearly a product of creative people working together! 🔝

[sakerlrf]

@dreamerlin, one more, please. Based on the step you recommended, I get the following, and then training is taking place:

unexpected key in source state_dict: conv1a.conv.weight, conv1a.conv.bias, conv2a.conv.weight, conv2a.conv.bias, conv3a.conv.weight, conv3a.conv.bias, conv3b.conv.weight, conv3b.conv.bias, conv4a.conv.weight, conv4a.conv.bias, conv4b.conv.weight, conv4b.conv.bias, conv5a.conv.weight, conv5a.conv.bias, conv5b.conv.weight, conv5b.conv.bias, fc6.weight, fc6.bias, fc7.weight, fc7.bias

missing keys in source state_dict: backbone.conv1a.conv.weight, backbone.conv1a.conv.bias, backbone.conv2a.conv.weight, backbone.conv2a.conv.bias, backbone.conv3a.conv.weight, backbone.conv3a.conv.bias, backbone.conv3b.conv.weight, backbone.conv3b.conv.bias, backbone.conv4a.conv.weight, backbone.conv4a.conv.bias, backbone.conv4b.conv.weight, backbone.conv4b.conv.bias, backbone.conv5a.conv.weight, backbone.conv5a.conv.bias, backbone.conv5b.conv.weight, backbone.conv5b.conv.bias, backbone.fc6.weight, backbone.fc6.bias, backbone.fc7.weight, backbone.fc7.bias, cls_head.fc_cls.weight, cls_head.fc_cls.bias

Could you please indicate at which area (script), should the missing keys be specified? Where to find or set the state_dict?

Thanks a lot!

[dreamerlin]

The issue is casued by mis-match ckpt. The sports1m one if for backbone finetune, and the load_from should be pass pth on the whole model.

1. If you finetune on sports1m, pretrained='https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_pretrain_20201016-dcc47ddc.pth'
2. If you finetune on ucf101, load_from=https://download.openmmlab.com/mmaction/recognition/c3d/c3d_sports1m_16x1x1_45e_ucf101_rgb/c3d_sports1m_16x1x1_45e_ucf101_rgb_20201021-26655025.pth

[sakerlrf]

Serious mistake made.

Thank you @dreamerlin.