Bayesian Optimization

HEBO (Heteroscedastic and Evolutionary Bayesian Optimisation)

Class or Function Names HEBOSampler Installation pip install -r https://hub.optuna.org/samplers/hebo/requirements.txt git clone git@github.com:huawei-noah/HEBO.git cd HEBO/HEBO pip install -e . Example search_space = { "x": FloatDistribution(-10, 10), "y": IntDistribution(0, 10), } sampler = HEBOSampler(search_space) study = optuna.create_study(sampler=sampler) See example.py for a full example. Others HEBO is the winning submission to the NeurIPS 2020 Black-Box Optimisation Challenge. Please refer to the official repository of HEBO for more details. Reference Cowen-Rivers, Alexander I., et al.

PFNs4BO sampler

Class or Function Names PFNs4BOSampler Installation pip install -r https://hub.optuna.org/samplers/pfns4bo/requirements.txt Example from __future__ import annotations import os import optuna import optunahub module = optunahub.load_module("samplers/pfns4bo") PFNs4BOSampler = module.PFNs4BOSampler def objective(trial: optuna.Trial) -> float: x = trial.suggest_float("x", -10, 10) return (x - 2) ** 2 if __name__ == "__main__": study = optuna.create_study( sampler=PFNs4BOSampler(), ) study.optimize(objective, n_trials=100) print(study.best_params) print(study.best_value) See example.py for a full example. You need GPU to run this example. The following figures are experimental results of the comparison between PFNs4BO and the random search.

PLMBO (Preference Learning Multi-Objective Bayesian Optimization)

Class or Function Names PLMBOSampler Installation pip install -r https://hub.optuna.org/samplers/plmbo/requirements.txt Example from __future__ import annotations import matplotlib.pyplot as plt import optuna import optunahub from optuna.distributions import FloatDistribution import numpy as np PLMBOSampler = optunahub.load_module( # type: ignore "samplers/plmbo", ).PLMBOSampler if __name__ == "__main__": f_sigma = 0.01 def obj_func1(x): return np.sin(x[0]) + x[1] def obj_func2(x): return -np.sin(x[0]) - x[1] + 0.1 def obs_obj_func(x): return np.array( [ obj_func1(x) + np.random.normal(0, f_sigma), obj_func2(x) + np.

SMAC3

Class or Function Names SAMCSampler Installation pip install -r https://hub.optuna.org/samplers/smac_sampler/requirements.txt Example import optuna import optunahub module = optunahub.load_module("samplers/smac_sampler") SMACSampler = module.SMACSampler def objective(trial: optuna.trial.Trial) -> float: x = trial.suggest_float("x", -10, 10) y = trial.suggest_int("y", -10, 10) return x**2 + y**2 n_trials = 100 sampler = SMACSampler( { "x": optuna.distributions.FloatDistribution(-10, 10), "y": optuna.distributions.IntDistribution(-10, 10), }, n_trials=n_trials, ) study = optuna.create_study(sampler=sampler) study.optimize(objective, n_trials=n_trials) print(study.best_trial.params) See example.py for a full example. Others SMAC is maintained by the SMAC team in automl.