« Back to top page

Hyperparameter Optimization

Hyperparameter Optimization

Differential Evolution with Hyperband (DEHB) Sampler

Class or Function Names DEHBSampler DEHBPruner Installation There is no additional installation required for this sampler and pruner, but if you want to run the example.py script, you need to install the following packages: $ pip install sklearn Example sampler = DEHBSampler() pruner = DEHBPruner(min_resource=1, max_resource=n_train_iter, reduction_factor=3) study = optuna.create_study(sampler=sampler, pruner=pruner) See example.py for a full example. The following figures are obtained from the analysis of the optimization. Others References Awad, N.

HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO

Abstract Hyperparameter optimization benchmark introduced in the paper HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each configuration. APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use. It must be in the range of [0, 7].

HPOLib: Tabular Benchmarks for Hyperparameter Optimization and Neural Architecture Search

Abstract Hyperparameter optimization benchmark introduced in the paper Tabular Benchmarks for Hyperparameter Optimization and Neural Architecture Search. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each configuration. APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use. It must be in the range of [0, 3].

NATS-Bench (NAS-Bench-201): Benchmarking NAS Algorithms for Architecture Topology and Size

Abstract Neural architecture search benchmark introduced in the paper NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each architecture. The preliminary version is the NAS-Bench-201, but since the widely used name is NAS-Bench-201, we stick to the name, NAS-Bench-201 APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use.