How to use pytorch-lightning for meta learning

[wheatdog]

Thanks for making this great library! Appreciate your work very much!

I wonder if anyone has the experience to use pytorch-lightning for meta-learning, maybe combined with library like Torchmeta or learn2learn.

I would like to hear from others before getting my hands dirty. Thanks!

[williamFalcon]

could you post a toy example?

[williamFalcon]

@wheatdog will reopen if we need to add to examples

[pietrolesci]

Hi @wheatdog,

I am working on meta-learning. I personally prefer the higher library as it gives more flexibility and is "nearer to the metal". I tried to reimplement this example using pytorch-lightning. It still need some nurturing but the high-level (design) idea is there. You can find my implementation here.

In brief, I matched the meta-training steps to the PL hooks in the following way

• Data loading: delegate all the meta-training batch loading to a custom DataLoader. I highly recommend using TorchMeta for this. You can then return it in the training_dataloader of a DataModule. Note: each batch will be a set of tasks

• Meta-training Implement all the meta-training logic in the training_step: each batch is a "batch of tasks". Loop on it and compute the inner gradients in the higher context. You have to take control of the optimization step so that PL does not do it automatically. Thus you need to

  • (i) disable automatic optimization in Trainer: Trainer(automatic_optimization=False)

  • (ii) drop or ignore the optimizer_idx argument

  • (iii) use self.manual_backward(loss) instead of loss.backward() to automatically scale your loss

more info available here

• Optimizers: let the configure_optimizer return the meta_optimizer and the inner_optimizer. You can access them in the training_step via self.optimizers()

• Meta-testing: same as meta-training, your dataloader should return a batch of tasks.

[GilbertoCunha]

I just checked your code since I've run into some troubles making this work with lightning. I have a question about your implementation, though. In the training_step of your MAML class, aren't you updating the model parameters when you call diffopt.step in the inner loop? Shouldn't you just be updating those parameters outside of the loop (where you seem to only accumulate the gradients using self.manual_backward, and not update the parameters) ?

Sorry for my confusion and thanks in advance!

[WHan-alter]

Hi @pietrolesci ,

I checked your code, and found that higher does not support pytorch-lightning optimizers. How do you deal with this issue?

[pietrolesci]

Hi @wakaizen,

Thanks for your reply. Maybe I don't get your question. Aren't you able to run the code?

Also, can you please clarify what do you mean by "pytorch-lightning optimizers"? The configure_optimizer returns a pytorch optimizer

[WHan-alter]

@pietrolesci I run your code, and got such errors:

File "/home/hanw/miniconda3/envs/meta_learning/lib/python3.7/site-packages/higher/optim.py", line 786, in get_diff_optim
    "Optimizer type {} not supported by higher yet.".format(type(opt))
ValueError: Optimizer type <class 'pytorch_lightning.core.optimizer.LightningSGD'> not supported by higher yet.

It seems that during training, the PyTorch SGD optimizer is converted to the Lightning optimizer, and higher does not support such optimizer since it's not differentiable.
By the way, I am using PyTorch 1.4, the latest lightning and the latest higher.

[ojss]

Did you figure out a way to avoid this?

@williamFalcon sorry to bother you, but is there a way to use the vanilla torch optimizers?

[WHan-alter]

Anyway, it seems to be a long way to go before combining the two. Since now higher can only support 1 GPU. It can not handle DP or DDP...

[chris-tkinter]

Is there a way to integrate learn2learn with pytorch lightning? I tried to convert this this simple MAML script into lightning, and this is what I tried so far. However I do have some errors about gradients...

[ojss]

What're the errors you receive?

[chris-tkinter]

RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

It seems that it is due to learn2learn's adapt method calls the

grad(loss,
         diff_params,
         retain_graph=second_order,
         create_graph=second_order,
         allow_unused=allow_unused)

where diff_params is the set of parameters that has requires_grad = True. However I think pytorch lightning automatically convert the parameters to be requires_grad = False (although they are still leaf variable), so having the above error.

[pietrolesci]

@chris-tkinter there seems to be something in the making from the folks at learn2learn that supports lightning. Haven't tried it yet, though

https://github.com/learnables/learn2learn/blob/master/learn2learn/utils/lightning.py

[GilbertoCunha]

From what I know, Lightning still does not support higher order differentiation because of its optimizers.

…

________________________________ From: chris-tkinter ***@***.***> Sent: Monday, July 5, 2021 1:44:26 AM To: PyTorchLightning/pytorch-lightning ***@***.***> Cc: Gilberto Rui Nogueira Cunha ***@***.***>; Comment ***@***.***> Subject: Re: [PyTorchLightning/pytorch-lightning] How to use pytorch-lightning for meta learning (#678) Is there a way to integrate learn2learn<https://github.com/learnables/learn2learn> with pytorch lightning? I tried to convert this this simple MAML script<https://github.com/learnables/learn2learn/blob/master/examples/vision/maml_omniglot.py> into lightning, and this<https://gist.github.com/chris-tkinter/74bfe9ff1237b9a938bb8345f410a23a> is what I tried so far. However I do have some errors about gradients... — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#678 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AN7Z5SFFLK2RV4HXT32JSLTTWD56VANCNFSM4VLBSFIA>.

[bmahlbrand]

Is there any progress on lightning being a good fit for metalearning?

[tchaton]

Dear people,

We PoCed an external loop which could be used to orchestrate task sampling and multiple trainer fit calls. I wondered if this sounds like the right path forward: #8715

You might also want to have a look at Sequoilla: https://github.com/lebrice/Sequoia/blob/9ac75fdbf6819b02b9f9ef64fc2a6c81290b62ff/examples/basic/pl_example.py

Best,
T.C

[brando90]

I plan to use this:

https://github.com/learnables/learn2learn/blob/master/examples/vision/distributed_maml.py

see: learnables/learn2learn#263