Hardware for following and reproduction

[momo1986]

Thanks for sharing.

If I want to follow this type of work, what is the basic hardware requirement for training?

Thanks & regards!
Momoi

[zw615]

@momo1986 Hi, our pre-training on web-scale data was done on TPU v3-256. As for fine-tuning on ImageNet, it is a more lightweight process which I believe can be done with only 32 or even 16 TPU/GPUs. You may want to check the Table 5 in our paper for comparison of compute across different methods.