RuntimeWarning: Optimization failed in gen_candidates_scipy / New set of initial conditions / OptimizationWarning failed within scipy.optimize.minimize

[MoBurmeister]

Hello together,

I am currently running different optimization problems within botorch. When running an optimization loop I often come accross the following warnings:

anaconda3\envs\TL_GP\lib\site-packages\botorch\optim\optimize.py:367: RuntimeWarning: Optimization failed in gen_candidates_scipy with the following warning(s):
[NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal'), NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal'), OptimizationWarning('Optimization failed within scipy.optimize.minimize with status 2 and message ABNORMAL_TERMINATION_IN_LNSRCH.'), NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal')]
Trying again with a new set of initial conditions.

C:\ProgramData\anaconda3\envs\TL_GP\lib\site-packages\botorch\optim\optimize.py:367: RuntimeWarning: Optimization failed in gen_candidates_scipy with the following warning(s):
[NumericalWarning('A not p.d., added jitter of 1.0e-08 to the diagonal'), OptimizationWarning('Optimization failed within scipy.optimize.minimize with status 2 and message ABNORMAL_TERMINATION_IN_LNSRCH.'), NumericalWarning('A not p.d., added jitter of 1.0e-08 to the
diagonal')]
Trying again with a new set of initial conditions.
warnings.warn(first_warn_msg, RuntimeWarning)
C:\ProgramData\anaconda3\envs\TL_GP\lib\site-packages\botorch\optim\optimize.py:389: RuntimeWarning: Optimization failed on the second try, after generating a new set of initial conditions.
warnings.warn(

As far as I can tell these warnings arise, when in the optimize_acqf method, when feasible candidates cannot be computed based on the set of initial conditions in the multi-start optimization. But I am not really familiar with what exactly is going on under the hood.

Here is some sample code with the provided VehicleSafety problem in botorch, where I encounter these warnings.

import torch
import botorch
from botorch.models import SingleTaskGP
from botorch.models.transforms.outcome import Standardize
from botorch.models.transforms.input import Normalize
from gpytorch.mlls.exact_marginal_log_likelihood import ExactMarginalLogLikelihood
from botorch.fit import fit_gpytorch_mll
from botorch.optim import optimize_acqf
from botorch.acquisition.multi_objective.logei import qLogNoisyExpectedHypervolumeImprovement
from botorch.test_functions.multi_objective import VehicleSafety

bounds = torch.tensor([[1.0, 1.0, 1.0, 1.0, 1.0], [3.0, 3.0, 3.0, 3.0, 3.0]])

train_X = torch.rand(10, 5, dtype=torch.float64) * (bounds[1] - bounds[0]) + bounds[0]
function = VehicleSafety()

def evaluate_train_y(x):
    # Evaluate the train_Y values for the given x tensor
    y = function.evaluate_true(x)
    y = -y
    return y

train_Y = evaluate_train_y(train_X)

#setup gp
model = SingleTaskGP(train_X, train_Y, outcome_transform=Standardize(m=3), input_transform=Normalize(d=5))

mll = ExactMarginalLogLikelihood(likelihood=model.likelihood, model=model) 
fit_gpytorch_mll(mll)

def perform_optimization_step(model, train_X, train_Y, bounds, num_restarts=10, raw_samples=512):
    
    qNEHVI = qLogNoisyExpectedHypervolumeImprovement(
        model=model,
        ref_point=torch.tensor([-1864.72022, -11.81993945, -0.2903999384]),
        X_baseline=train_X,
        prune_baseline=True
    )

    # Optimize the acquisition function to find the next point to sample
    new_X, acq_func_value = optimize_acqf(
        acq_function=qNEHVI,
        bounds=bounds,
        q=1,
        num_restarts=num_restarts,
        raw_samples=raw_samples,
        options={"batch_limit": 5, "maxiter": 200},
    )

    print(f"Acquisition function value: {acq_func_value}")
    
    return new_X, acq_func_value

def optimize_vehicle_safety(train_X, train_Y, bounds, num_iterations=100):
    # Setup GP model
    model = SingleTaskGP(train_X, train_Y, outcome_transform=Standardize(m=3), input_transform=Normalize(d=5))
    mll = ExactMarginalLogLikelihood(likelihood=model.likelihood, model=model)
    fit_gpytorch_mll(mll)
    
    for iteration in range(num_iterations):
        print(f"Iteration {iteration + 1}/{num_iterations}")
        new_X, acq_func_value = perform_optimization_step(model, train_X, train_Y, bounds)
        
        # Evaluate new Y values
        new_Y = evaluate_train_y(new_X)
        
        # Update training data
        train_X = torch.cat([train_X, new_X], dim=0)
        train_Y = torch.cat([train_Y, new_Y], dim=0)
        
        # Refit model with updated data
        model = SingleTaskGP(train_X, train_Y, outcome_transform=Standardize(m=3), input_transform=Normalize(d=5))

        mll = ExactMarginalLogLikelihood(likelihood=model.likelihood, model=model) 
        fit_gpytorch_mll(mll)
    
    return train_X, train_Y

train_X, train_Y = optimize_vehicle_safety(train_X, train_Y, bounds)

The optimization works nevertheless and reaches good results.
I mainly asked myself if I should be concerned by these warnings and if there are potentially errors/wrong settings in my code, which can cause this behaviour.
Especially the setup of the optimize_acqf is a bit unclear for me. If I raise the num_restarts and the raw_samples, this will potentially result in less warnings but comes at a computational cost? What are good values for these parameters and how should I set these in regards to the characteristics of my optimization problem?

I know that running into numerical problems is common when fitting a surrogate model, especially in more demanding optimization settings with mulitple objectives.
But maybe there are still some good tips and best practices when using botorch, which can further enhance the optimization behavior of my code and can also potentially help others :)

Many thanks in advance for any help or comments!

[Balandat]

The optimization works nevertheless and reaches good results.

Great :)

I mainly asked myself if I should be concerned by these warnings and if there are potentially errors/wrong settings in my code, which can cause this behaviour.

I don't see anything wrong with your code. Fundamentally, acquisition function optimization is a non-convex problem; we're using L-BFGS-B under the hood to solve this. Depending on how non-convex the problem is the line search in that method can fail. If the overall performance is good this is nothing to be too concerned about. I'm not sure about the reference point you're using and how much data you have where the observations exceed that reference point, if this is selected very aggressively then backing off there could help make the problem more numerical well-behaved.

I know that running into numerical problems is common when fitting a surrogate model, especially in more demanding optimization settings with mulitple objectives.

Indeed. One would have to go pretty deep and invest quite a bit of time to understand what exactly is going on in your specific case.

Especially the setup of the optimize_acqf is a bit unclear for me. If I raise the num_restarts and the raw_samples, this will potentially result in less warnings but comes at a computational cost? What are good values for these parameters and how should I set these in regards to the characteristics of my optimization problem?

Yeah, this is a good question. Increasing num_restarts and raw_samples will both make it more likely to find the global optimum in the non-convex optimization problem. Increasing num_restarts will be more costly. How exactly performance depends on the two depends on the acquisition function surface you're optimizing. A general rule of thumb is that you it makes sense to increase both so long as it doesn't slow down your optimization too much. It would be good to have a more comprehensive study of this across a number of different use cases, but we haven't had the bandwidth to do this.

[MoBurmeister]

Thanks for the quick response!

I will play around a bit with the settings for the optimize_acqf function and see what happens. If I have time, I will report some findings here in this thread.

Another question i came accross when playing around with my code: When should I be putting the model into model.eval() mode and what exact benefits does it have regarding the optimization loop in botorch?
I know it from the use in pytorch, where it is helpful regarding the dropout and the batch normalization, but how much does the optimization in botorch benefit from it.
For the potential use: I would call model.eval() before computing the acquisition function?
From the documentation I found: what does it mean, that model.eval() sets the transformed inputs?

And lastly, should I also call mll.eval()?

Thanks again for all the help! :)

[Balandat]

Another question i came accross when playing around with my code: When should I be putting the model into model.eval() mode and what exact benefits does it have regarding the optimization loop in botorch? I know it from the use in pytorch, where it is helpful regarding the dropout and the batch normalization, but how much does the optimization in botorch benefit from it. For the potential use: I would call model.eval() before computing the acquisition function?

module.train() and module.eval() are not really things you need to worry about when using botorch (only when you want to change things deep in the botorch stack itself). As you know they are concepts from pytorch, which in gpytorch essentially correspond to whether a GP model is in "prior" mode or in "posterior" mode - i.e. if you evaluate the GP prior or the GP posterior (conditioned on some data). The botorch helpers handle this under the hood (e.g. making sure the underlying gpytorch model is in eval() mode when calling posterior(), so you don't need to worry about this.

From the documentation I found: what does it mean, that model.eval() sets the transformed inputs?

It's the mechanism under the hood that applies input transformations to the training data. This is also something you don't need to worry about during regular botorch usage.

And lastly, should I also call mll.eval()?

No, this also happens automatically under the hood if you use fit_gpytorch_mll.

[MoBurmeister]

Thanks a lot for the clarifications!