Callback decorator state

[kellertuer]

While the main design decision in Manopt are decorators, it might be nice to have Callbacks as well.
With the modularity of Manopt, it is maybe a bit tricky to do these. Here is an idea, that could even be used in debug/record then as well.

An algorithm declares “Callback anchors” within the solver_step! function – maybe using a macro like

@callback_anchor :AnchorName

by default this generates a no-op call to the “callback-dispatch” or in other words “registers” the anchor.
By default this dispatch would just be a no-op (maybe something like callback(Val(:AnchorName), problem, state, iteration)
An CallbackState could then be a decorator with a dictionary of callbacks for anchors.

This would allow anchors for example like

• :BeforeSubsolver
• :AfterSubSolver

or any other reasonable phase within one step of the solver. The default anchors could already be the ones debug/record themselves have (before/after iteration, before/after algorithm)

The signature of a callback function would hence always be cbf(problem, state, iteration), though one could probably also allow “simple” ones as cbfs().

On the high-level interfaces, this would take the same route as debug/record and one could maybe specify them as pairs

callbacks = [ :AnchorName => cbf1, :AnotherAnchorName => cfb2]