Ablation-study of model

[texhnolyze]

Summary

To properly evaluate effects of changes on performance of the model we want to setup a proper ablation study.
Current ideas what to evaluate are:

• different embeddings
  • angle represetations for IMU data (e.g. quaternion vs euler-angles)
  • different vision encoders (e.g. ResNet vs DINO)
• different hyperparameters
  • context length of sequences
  • number of prediction steps (timeframe) into the future
  • Robot type (on/off)
  • hidden_dim, num_decoder_layers, *encoder_layers
• datasets
  • wolfgang / wolfgang + nao
  • with / without simulation