[WIP] L-BFGS approximation for SQP methods

[cvanaret]

Support for L-BFGS.

Limited memory quasi-Newton Hessian approximations will be computed based on the compact QN representation (see Numerical optimization by Nocedal & Wright, pp 181-184).

@worc4021 @FrancoisGallard @leyffer

Builds upon and supersedes #150.

• uses BLAS and LAPACK
• if no Hessian is provided for QPs and NPLs, pick L-BFGS Hessian

[damienhocking]

Excellent

[cvanaret]

@amontoison
Uno-filtersqp with a line search and an L-BFGS Hessian seems to run ✌️🎉
The code is neither very clean nor clever, I'll check it on Monday before I merge.

[amontoison]

@cvanaret We already require BLAS in Uno, we should do the same thing with LAPACK.
Many libraries contain the symbols for both like MKL, OpenBLAS, ...

[amontoison]

@cvanaret In the C interface, do you use LBFGS as fallback if the Hessian of the Lagrangian is not available?
Should we do that if the users didn't set hessian_model=LBFGS?

[cvanaret]

@cvanaret We already require BLAS in Uno, we should do the same thing with LAPACK. Many libraries contain the symbols for both like MKL, OpenBLAS, ...

At the moment, BLAS is only required if one of the linear solvers (HSL or MUMPS) is linked. Obviously it's always the case for UnoSolver.jl and unopy, but any C/C++ user who compiles with CMake might not link them. Should we systematically require BLAS and LAPACK anyway?

[cvanaret]

@cvanaret In the C interface, do you use LBFGS as fallback if the Hessian of the Lagrangian is not available? Should we do that if the users didn't set hessian_model=LBFGS?

I don't do it yet, but it's the plan yes!
At the moment we use a zero Hessian:

Uno/uno/ingredients/hessian_models/HessianModelFactory.cpp

Line 26 in 3c2eeae

return std::make_unique<ZeroHessian>(model.number_variables);

Edit: done

[amontoison]

@cvanaret We already require BLAS in Uno, we should do the same thing with LAPACK. Many libraries contain the symbols for both like MKL, OpenBLAS, ...

At the moment, BLAS is only required if one of the linear solvers (HSL or MUMPS) is linked. Obviously it's always the case for UnoSolver.jl and unopy, but any C/C++ user who compiles with CMake might not link them. Should we systematically require BLAS and LAPACK anyway?

Yes, I think we should require them. We can rely more on it in the future this way. For example, the dot product can always be the one implemented in BLAS.

[cvanaret]

OK, I solved the failing workflows (C and Fortran) by linking LAPACK and BLAS.

[cvanaret]

Now I'm pretty proud of myself. I've implemented operator overloading/symbolic computations such that I can write the computation U = (δ S + Y D⁻¹ Lᵀ) J⁻ᵀ as:

Uk = Sk;
Uk = this->delta * Uk + Vk * transpose(L_invsqrt_Dk);
Uk *= transpose(inverse(Jk));

and BLAS is called under the hood (copy, gemm, and trsm). Dimensions and leading dimensions are handled automatically.
Same for Cholesky (potrf) and low-rank update (syrk).
🤩

[amontoison]

@cvanaret Can you also add a small test for LBFGS in Fortran and Julia interfaces?

• https://github.com/cvanaret/Uno/blob/main/interfaces/Fortran/example_uno.f90
• https://github.com/cvanaret/Uno/blob/main/interfaces/Julia/test/NLP_wrapper.jl

[cvanaret]

@cvanaret Can you also add a small test for LBFGS in Fortran and Julia interfaces?

* https://github.com/cvanaret/Uno/blob/main/interfaces/Fortran/example_uno.f90

* https://github.com/cvanaret/Uno/blob/main/interfaces/Julia/test/NLP_wrapper.jl

@amontoison
Done! I must have made a mistake (but which one?) when defining the test in NLP_wrapper.jl, I get:

Error During Test at /home/runner/work/Uno/Uno/interfaces/Julia/test/NLP_wrapper.jl:37
  Got exception outside of a @test
  CUTEst.CUTEstException(1, "memory allocation error")

[amontoison]

You need the finalize the model if you recreate a CUTEstModel from the same problem.
You have an error because we don't dlopen / dlclose the shared library for the CUTEst problem the second time.

[cvanaret]

@amontoison thanks, that's a wrap 😎