Abstract When running parallel optimization experiments using tabular or surrogate benchmarks, each evaluation must be ordered based on the runtime that each configuration would take in reality. However, the evaluation of tabular or surrogate benchmarks, by design, does not take long. For this reason, the timing of each configuration must be ordered as if we actually evaluated each configuration.
This package provides a simulator that automatically handles this problem by internally managing the order of hyperparameter configuration evaluations.
Class or Function Names GPEISampler (alias for optuna.samplers.GPSampler) GPPISampler GPUCBSampler GPTSSampler Installation pip install scipy torch Overview Optuna’s built-in GPSampler only supports Expected Improvement (EI) as an acquisition function. This package extends GPSampler with three additional acquisition functions commonly used in Bayesian optimization:
Sampler Acquisition Function Description GPEISampler Expected Improvement (EI) Alias for optuna.samplers.GPSampler. Balances improvement magnitude and probability. GPPISampler Probability of Improvement (PI) Selects the point most likely to improve over the current best.
Class or Function Names LevyFlightSampler Overview This sampler proposes new hyperparameter candidates by taking a Lévy-flight step from the current best trial. A Lévy flight is a random walk whose step lengths follow a heavy-tailed (Lévy stable) distribution: most steps are small, enabling fine local search, but occasional large steps jump far away, helping to escape shallow local optima.
The step is computed via the Mantegna algorithm, an efficient closed-form approximation of the Lévy stable distribution that requires only standard Gaussian random variates.
Abstract This visualization module enables users to plot the target value over time with standard error bands. This module is especially convenient when we use parallel optimization such as Asynchronous optimization simulation.
Class or Function Names plot_target_over_time Installation This module requires the following dependencies:
matplotlib numpy APIs plot_target_over_time(study_list, *, ax=None, states=None, target=None, target_direction=None, cumtime_func=None, log_time_scale=True, n_steps=100, color=None, **plot_kwargs) study_list: A list of optuna.Study objects. Each study is treated as one run, and results are averaged across them.