Abstract When running parallel optimization experiments using tabular or surrogate benchmarks, each evaluation must be ordered based on the runtime that each configuration would take in reality. However, the evaluation of tabular or surrogate benchmarks, by design, does not take long. For this reason, the timing of each configuration must be ordered as if we actually evaluated each configuration.
This package provides a simulator that automatically handles this problem by internally managing the order of hyperparameter configuration evaluations.
Abstract BatchSampler solves a coordination problem that arises when running Optuna with multiple workers (n_jobs > 1).
With default samplers, each worker calls sample_relative concurrently and sees the same incomplete view of the trial history — pending trials from the same batch are invisible to one another. This causes workers to suggest near-duplicate configurations, wasting the throughput gained from parallelism.
BatchSampler uses a shared lock to coordinate workers: the first worker to find an empty cache calls a user-supplied suggest_fn once to obtain q suggestions jointly, then hands them out one at a time.
Abstract This package provides a parallel coordinate plot for Optuna studies. It follows Optuna’s visualization module layout: plot_parallel_coordinate returns a Plotly figure, and matplotlib.plot_parallel_coordinate returns a Matplotlib axes object.
The current implementation adds:
Conditional parameter support: if a trial does not contain a selected parameter, the line is connected to a special None tick below the valid values. Multi-objective support: objective axes are shown side by side, and trial colors are ordered by Pareto rank.
Class or Function Names GPEISampler (alias for optuna.samplers.GPSampler) GPPISampler GPUCBSampler GPTSSampler Installation pip install scipy torch Overview Optuna’s built-in GPSampler only supports Expected Improvement (EI) as an acquisition function. This package extends GPSampler with three additional acquisition functions commonly used in Bayesian optimization:
Sampler Acquisition Function Description GPEISampler Expected Improvement (EI) Alias for optuna.samplers.GPSampler. Balances improvement magnitude and probability. GPPISampler Probability of Improvement (PI) Selects the point most likely to improve over the current best.
Abstract HierarchicalTPESampler is a mixture-of-experts variant of Optuna’s multivariate, group-decomposed TPESampler for conditional search spaces (objectives where different trials request different parameters, e.g. if optimizer == "adam": ...).
Why this exists. On a conditional (dynamic) search space, Optuna’s own multivariate TPESampler is inadequate. With group=False the conditional parameters fall back to independent (random) sampling, because a dynamic search space is not supported for multivariate=True. With group=True the space is decomposed into parameter groups that are each sampled independently, so correlation between groups is lost.
Abstract This package provides a utility to compute the hypervolume history of a multi-objective Optuna study. The hypervolume indicator measures the volume of the objective space dominated by the current Pareto front relative to a reference point, and tracking it over trials is a common way to evaluate the convergence of multi-objective optimizers. The implementation supports arbitrary numbers of objectives, mixed minimization/maximization directions, and constraint handling via Optuna’s constraint mechanism. For 2D and 3D cases, specialized efficient algorithms are used; for higher dimensions, the WFG algorithm is employed.
Class or Function Names LevyFlightSampler Overview This sampler proposes new hyperparameter candidates by taking a Lévy-flight step from the current best trial. A Lévy flight is a random walk whose step lengths follow a heavy-tailed (Lévy stable) distribution: most steps are small, enabling fine local search, but occasional large steps jump far away, helping to escape shallow local optima.
The step is computed via the Mantegna algorithm, an efficient closed-form approximation of the Lévy stable distribution that requires only standard Gaussian random variates.
Abstract Optuna’s built-in trial.report() raises NotImplementedError in multi-objective studies. MultiMetricPruner works around this by storing intermediate values in trial user attributes and constructing a synthetic single-objective study for the wrapped base pruner to evaluate.
The pruning mode is selected via the joint argument:
| Mode | `joint` | `report` call (Example with `metric_names = ["loss", "acc"]`) | | ------------ | ------- | ---------------------------------------------------------------------------------- | | Multi-metric | `True` | `trial.report({"loss": v1, "acc": v2}, step)` | | Per-metric | `False` | `trial.
Abstract Standard Bayesian optimization handles uncertainty by taking random samples of model parameters, but this random sampling creates noise. This noise can cause the optimizer to accidentally rank bad configurations above good ones and steer the search in the wrong direction. This issue is especially noticeable in complex tasks with many variables or when computational limits restrict the number of samples you can take.
This package implements OrthoBO, a framework introduced in this paper from May 2026.
Abstract This visualization module enables users to plot the target value over time with standard error bands. This module is especially convenient when we use parallel optimization such as Asynchronous optimization simulation.
Class or Function Names plot_target_over_time Installation This module requires the following dependencies:
matplotlib numpy APIs plot_target_over_time(study_list, *, ax=None, states=None, target=None, target_direction=None, cumtime_func=None, log_time_scale=True, n_steps=100, color=None, **plot_kwargs) study_list: A list of optuna.Study objects. Each study is treated as one run, and results are averaged across them.
APIs A sampler that uses SMAC3 v2.4.0 verified by unittests that can be run by the following:
$pip install pytest -r https://hub.optuna.org/samplers/smac_sampler/requirements.txt $ python -m pytest package/samplers/smac_sampler/tests/ Please check the API reference for more details:
https://automl.github.io/SMAC3/main/5_api.html SMACSampler(search_space: dict[str, BaseDistribution], n_trials: int = 100, seed: int | None = None, *, surrogate_model_type: str = "rf", acq_func_type: str = "ei_log", init_design_type: str = "sobol", surrogate_model_rf_num_trees: int = 10, surrogate_model_rf_ratio_features: float = 1.0, surrogate_model_rf_min_samples_split: int = 2, surrogate_model_rf_min_samples_leaf: int = 1, init_design_n_configs: int | None = None, init_design_n_configs_per_hyperparameter: int = 10, init_design_max_ratio: float = 0.