JSO-compliance definition

[abelsiqueira]

JSO-compliant definition is currently very loose:

A JSO-compliant solver is a solver whose

• input is a model implementing the NLPModels API; and
• output is a specific struct from the package SolverCore.

Source: https://jso.dev/tutorials/creating-a-jso-compliant-solver/

But we assume some additional things from our solvers, which we should include to JSO-compliance, at least as optional but desired features.

We should also define this somewhere and always link back to it. Most mentions of JSO-compliance outside of that blog don't explicitly define it.
Some formal description would be useful.

Here are some things I think are already in place:

• Implement SolverName <: SolverCore.AbstractOptimizationSolver. Maybe more specifically Solver{T, V, ...}, for type T and vector type V?
• Implement SolverName(nlp; kwargs...)
• Implement solve!(solver, nlp, stats; kwargs...) such that:
  • It makes no allocations
  • It allows the keyword arguments x, atol, rtol, max_eval, max_iter, max_time, callback, verbose.
  • The function callback = (nlp, solver, stats) -> nothing is called right before the end of the loop.
• Implements solver_name(nlp; kwargs...) that calls SolverName(nlp; kwargs...) and then solve!(solver, nlp; kwargs...)
• reset!(solver), which resets specific things (maybe not part of compliance)
• reset!(solver, nlp), which is also available, I think because of QuasiNewtonModels? (maybe not part of compliance)

Here are some suggestions:

• SolverName should have a field x::V for the iterate.
• Change to solve!(stats, nlp, solver), since the stats will be modified (although solver is modified too, it is not an "output", in the sense that all stored data should not be trusted).

Questions/Issues:

• Store output inside solver? It is starting to feel too OOP.
• x is currently stored inside solver and output (called solution there). I say we move it to output only, specially since solve! now requires stats.
• reset! is only useful for the solver itself, right? Probably shouldn't be part of compliance if that is the case.
• What should happen if I call solve!(stats, nlp, solver) twice in a row (no kwargs).
• Where should we store this definition?
• Does Krylov need a similar definition?

[tmigot]

Hi @abelsiqueira !

I think we should have a place with a detailed tutorial that goes beyond the "takes an NLP, returns a Stats", what do you think? We could do it as a JSOTutorials.jl and maybe have a template for JSO-compliant solver !?
It would also help clarify the generic unit tests.

I think you have a quite complete list of what exists. The only update I would see is adding a structure for the parameters.

Essentially, we wait 3 things for a JSO-compliant solver:

• Basic solve: stats = mysolver(nlp)
• In-place solve: solve!(MySolverStruct, nlp, stats)
• Re-solve: solve!(MySolverStruct, nlp, stats); reset!(MySolverStruct); solve!(MySolverStruct, nlp, stats)
• (opt) You could also re-solve with a different nlp but with similar characteristics: solve!(MySolverStruct, nlp, stats); reset!(MySolverStruct, nlp); solve!(MySolverStruct, nlp, stats)

This behavior is tested, but maybe not extensively documented.

About your suggestion of Change to solve!(stats, nlp, solver), I am fine with it. Feel free to open an issue or a PR on SolverCore and I can help propagate this.

I am not too fond of storing the output inside the solver, but this is what is done in Krylov.jl. It has the merit of avoiding storing twice the solution.

[abelsiqueira]

That's a great idea. I was thinking of defining it; creating tests for all of it (test_minimum_compliance(::Type{Solver}), test_in_place_compliance..., test_keywords_compliance..., etc.; updating a few solver packages with it; creating the tutorial.
Also, I've started gathering these in the https://github.com/JuliaSmoothOptimizers/JSOSolverSkeleton.jl/, which I updated to my talk using one of our solvers.

I agree with the "output inside solver" part. I don't like it so much, and I don't want to make large changes.

[tmigot]

Oh, I missed the JSOSolverSkeleton.jl. Maybe this package could contain comments with further details on each parts.
I will open a couple of issues there, to move the discussion to something more concrete.