Abstract This callback implements an automatic stopping mechanism for Optuna studies, aiming to avoid unnecessary computation. The optimization is terminated when the statistical error of the objective function (e.g., cross-validation error) exceeds the room left for optimization (i.e., the estimated potential for improvement).
The mechanism is described in the following papers:
A. Makarova et al. Automatic termination for hyperparameter optimization. <https://proceedings.mlr.press/v188/makarova22a.html>__ H. Ishibashi et al. A stopping criterion for Bayesian optimization by the gap of expected minimum simple regrets.
Installation pip install trackio Abstract This callback enables tracking of Optuna studies in Trackio. By default, the study is tracked as a single experiment run, where all suggested hyperparameters and optimized metrics are logged and visualized as a function of optimizer steps.
Trackio is offline-first and does not require authentication for local experiment tracking. Optionally, tracked experiments can be synchronized to Hugging Face Spaces for remote visualization and sharing.
The callback also supports multi-run mode, where each Optuna trial is tracked as an independent Trackio run.
Abstract This callback enables tracking of Optuna study in Weights & Biases. The study is tracked as a single experiment run, where all suggested hyperparameters and optimized metrics are logged and plotted as a function of optimizer steps.
APIs WeightsAndBiasesCallback(metric_name: str | Sequence[str] = "value", wandb_kwargs: dict[str, Any] | None = None, as_multirun: bool = False) metric_name: Name assigned to optimized metric. In case of multi-objective optimization, list of names can be passed.