AIL_from_visual_obs_with_latent_information

Instructions

Use anaconda to create a virtual environment

Step 1. Install miniconda

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Step 2. Install MuJoCo

Step 3. Clone repo and create conda environment

conda env create -f environment.yml
conda activate AdvIL_from_videos

Train expert

python train_expert.py task=walker_walk seed=0 agent=ddpg frame_skip=1

Create a new directory expert_policies, move the trained expert policy in expert_policies.

Alternatively, download the policies here and unzip in main directory.

Train imitation from experts

DAC

python train_w_expert_MDP.py task=walker_walk seed=0 GAN_loss=bce from_dem=true

DACfO

python train_w_expert_MDP.py task=walker_walk seed=0 GAN_loss=bce from_dem=false

patchAIL

python train_LAIL.py task=walker_walk agent=patchAIL seed=0 GAN_loss=bce discriminator_lr=1e-4

LAIfO

python train_LAIL.py task=walker_walk agent=lail seed=0 GAN_loss=bce from_dem=false

VMAIL

python train_VMAIL.py task=walker_walk seed=0 GAN_loss=bce from_dem=true

LAIL

python train_LAIL.py task=walker_walk agent=lail seed=0 GAN_loss=bce from_dem=true

Curves

The curves for experiments in the paper can be found in the curves repository.

Citation

If you use this repo in your research, please consider citing

@article{giammarino2023adversarial,
  title={Adversarial imitation learning from visual observations using latent information},
  author={Giammarino, Vittorio and Queeney, James and Paschalidis, Ioannis Ch},
  journal={arXiv preprint arXiv:2309.17371},
  year={2023}
}

Acknowledgements

This repo is based on the drqv2 repo.