How to solve the convergence problem of SNOPT

[haidejunyuan]

When I was conducting flow field inversion, I often encountered the following situation:

 SNOPTC EXIT  40 -- terminated after numerical difficulties
 SNOPTC INFO  41 -- current point cannot be improved

Here are some settings that I usually use. I'm not sure if they need to be modified?

    "primalMinResTol": 1.0e-7,
    "adjEqnOption": {
        "gmresRelTol": 1.0e-6,
        "gmresAbsTol": 1.0e-9,
        "pcFillLevel": 2,
        "jacMatReOrdering": "rcm",
        "gmresMaxIters": 8000,
        "gmresRestart": 8000,
    },
if args.optimizer == "SNOPT":
    prob.driver.opt_settings = {
        "Major feasibility tolerance": 1.0e-5,
        "Major optimality tolerance": 1.0e-5,
        "Minor optimality tolerance": 1.0e-5,
        "Minor feasibility tolerance": 1.0e-5,
        "Verify level": -1,
        "Function precision": 1.0e-5,
        "Major iterations limit": 150,
        "Linesearch tolerance": 0.99999,
        "Hessian updates": 10000,
        "Nonderivative linesearch": None,
        "Print file": "opt_SNOPT_print.txt",
        "Summary file": "opt_SNOPT_summary.txt",
    }

[friedenhe]

Yeah, this happens a lot, and one way to fix this is to randomize the initial design variables. Also, make sure the flow and adjoint converge well.

[haidejunyuan]

Can we use Adam as the optimizer? This might lead to better optimization results for neural networks. I'm not sure if embedding is complicated.

[friedenhe]

No, we can not because Adam is a stochastic optimizer and needs to call the gradient and primal solution many times; in CFD, we probably can not afford that...

[uttamcadambi07]

Actually I do face this quite a bit and I can give you some tips on what we do. As Dr. He mentioned, randomising the initial condition for the design variable space helps. Additionally, since you are solving a non-linear constrained optimisation problem (which is also non convex). For field inversion, however, the optimisation landscape is generally well behaved so an error such as current point cannot be improved genuinely means that the optimisation problem at that point is hard to solve (could also be because the gradients are very small which makes the next step negligibly better than the current one).

You could perturb the design variable at the point where you get the error and restart the optimisation. You can do a sensitivity study to see the extent of differences in convergence based on the magnitude of the perturbation. If, regardless of the perturbation, your solution converges to the same point, it can be somewhat safely declared that your solution is optimal.