Abstract Every built-in pruner is a heuristic, and none states how often it prunes a trial that would have ended well. This pruner does. You pick alpha, and the expected fraction of trials that were good and pruned stays at or below alpha, under one condition: the trials it calibrates on and the trials it judges must come from the same random process.
It runs the first n_calibration completed trials unpruned, calibrates a threshold on their learning curves by conformal risk control, then freezes and prunes.
Abstract This callback implements an automatic stopping mechanism for Optuna studies, aiming to avoid unnecessary computation. The optimization is terminated when the statistical error of the objective function (e.g., cross-validation error) exceeds the room left for optimization (i.e., the estimated potential for improvement).
The mechanism is described in the following papers:
A. Makarova et al. Automatic termination for hyperparameter optimization. <https://proceedings.mlr.press/v188/makarova22a.html>__ H. Ishibashi et al. A stopping criterion for Bayesian optimization by the gap of expected minimum simple regrets.