Pruning strategies for Optuna runs

[l3th4l]

Currently the optuna runs for findind the optimal hyper-parameters have no pruning strategies for "obviously useless" runs. The currently pruner works by:

• If the run returns a NaN value, the run is pruned.

We need to prune runs based on these conditions:

• If the losses start to diverge
• If an "unscaled" version of the loss function does no improve over certain epochs (alternatively, try only one of the loss terms if that helps)