Cyclic learning rates

[JohnGiorgi]

We should see if using the cyclic learning rate finder (paper: here) along with an adaptive learning rate optimizer (e.g., adam) improves on our current optimizer (nadam).

Todo

• Use the Cyclic LR Keras Callback to determine an optimal learning rate.
• Try this learning rate with a few different optimizers (starting with Adam). Does it beat our current optimization method?

Resources

• Cyclic LR paper
• Keras Callback for Cyclic LRs