Roadmap-ish stuff

[pkofod]

Documentation

• General QA on the contents
• Identify weak pages
• Add references

Solvers (up for grabs if the person is willing to "maintain")

• Subplex Feature Request: Subplex #272
• Parallel NelderMead Feature Request: parallel Nelder-Mead #268

Cleanup

• the new maximize helper function
• Fminbox api (still corner cases!)

Stolen from #326

[timholy]

I haven't followed Optim in ages (will dip back in occasionally when I need it for a new project or update an old one), so here's an incredibly naive "roadmap" question: what's the state of structured Hessians? E.g., for some problems it would be great to be able to provide the hessian in SparseMatrixCSC or Woodbury form. If this is not implemented, I'd be curious to know whether it's a valid roadmap item, since it's an area where the C libraries are limited (e.g., Ipopt doesn't support Woodbury at all) and Julia's composability could really shine and deliver some huge performance advantages.

[pkofod]

You should be able to do that. Can you show me a simple example?

[timholy]

Haven't tried. I just remember the days when it wasn't possible, and a brief search of both the README and the documentation failed to turn up any hits for "sparse." That's the extent of my investigations (sorry, this was me being lazy).

[pkofod]

I’ll try to give an example. Just gotta find a problem with a Woodbury matrix hessian :)

[pkofod]

I havn't found anything yet, but googling gives me Hessian Avenue in Woodbury, NJ. Not quite what I was looking for, but ... :)

[timholy]

Try

f(x) = sum(abs2, x)^2 + (a'*x)^2

for vector a and x. The Hessian is a Woodbury (well really, Sherman-Morrison) built on top of a diagonal matrix. With a Woodbury Hessian you should have no problem handling problems of scale

using WoodburyMatrices
a = rand(10^6)
W = SymWoodbury(Diagonal(ones(10^6)), a, 1.0)

# After warmup
julia> @time W \ rand(10^6);
  0.090865 seconds (23 allocations: 45.777 MiB, 75.45% gc time)

[bhalonen]

Anyone make any progress on paralell NelderMead? @pkofod

[pkofod]

Not really. Do you have a specific use-case for it? I don't think it's too hard to implement, I just happen to have gradients available for most of the stuff I do, so I rarely use "vanilla" nelder mead.

[bhalonen]

I am considering it as an option. I am working on a personal project of building a functional digital image correlation package, DIC.jl as a science project and a demonstration of capabilities for contracting Julia work. It works with Nelder Mead, but it takes about 45 seconds to finish for my demo case. I spent a few hours this morning building a gradient function. I made at least some progress on that, so I am going to pursue that for the short term.