Please write an example to find an optimal parameter using optuna.

import math

def function_to_optimize(x):
	assert x > 0
	assert x < 10

	return math.sin(x + 0.5)

# assuming that this function is much more complex, find the x that minimizes function_to_optimize for x in the range from 0 to 10.

Generate a slice plot at the end please.
?

comes highly recommended by Nestor. It is function agnostic.

It's surprisingly easy. The overhead is noticeable though. Also its choice of which parameters to try is not very transparent.

pip install optuna
pip install plotly
pip install nbformat # if you're running it in a jupyter notebook

import optuna
from optuna.visualization import plot_slice, plot_optimization_history

def function_to_optimize(x):
	assert x > 0
	assert x < 10

	return math.sin(x + 0.5)

def objective(trial):
	min_x = 0
	max_x = 10
    x_sugg = trial.suggest_float('x', min_x, max_x)
    abs_mean_error = function_to_optimize(x_sugg)

    return abs_mean_error

print("Hyperparameter search: this can take up to 3 minutes")
# Don't overthing the sampler, it just works and is efficient. It's meant for use with only one hyperparameter though.
study = optuna.create_study(direction='minimize', sampler=optuna.samplers.TPESampler())

# jobs = -1 to use all cores
study.optimize(objective, n_trials=50, n_jobs=-1, show_progress_bar=True)

print("best abs_mean_error: ", study.best_value)
print("best parameters: ", study.best_params)

To plot the values tried and the results:

plot_slice(study)

Pasted image 20250129225631.png

if you wish to pass parameters to the objective function:

study.optimize(lambda trial: objective_function(trial, param1=param1, param2=param2), n_trials=100, n_jobs=-1)