Abstract Hyperparameter optimization benchmark introduced in the paper HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each configuration.
APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use. It must be in the range of [0, 7].
Abstract Hyperparameter optimization benchmark introduced in the paper Tabular Benchmarks for Hyperparameter Optimization and Neural Architecture Search. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each configuration.
APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use. It must be in the range of [0, 3].
Abstract Neural architecture search benchmark introduced in the paper NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size. The original benchmark is available here. Please note that this benchmark provides the results only at the last epoch of each architecture.
The preliminary version is the NAS-Bench-201, but since the widely used name is NAS-Bench-201, we stick to the name, NAS-Bench-201
APIs class Problem(dataset_id: int, seed: int | None = None, metric_names: list[str] | None = None) dataset_id: ID of the dataset to use.
Abstract The blackbox optimization benchmarking (bbob) test suite comprises 24 noiseless single-objective test functions. BBOB is one of the most widely used test suites to evaluate and compare the performance of blackbox optimization algorithms. Each benchmark function is provided in dimensions [2, 3, 5, 10, 20, 40] with 110 instances.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use. It must be in the range of [1, 24].
Abstract The bbob-biobj test suite was created by combining existing 55 noiseless single-objective test functions. bbob-biobj (and its extension, bbob-biobj-ext) has in total of 92 (= original 55 + additional 37) bi-objective functions. Each benchmark function is provided in dimensions [2, 3, 5, 10, 20, 40] with 15 instances. In this package, all the 92 functions are available. Please refer to the paper for details about each benchmark function.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use.
Abstract The blackbox optimization benchmarking biobj-mixed-integer (bbob-biobj-mixint) test suite comprises 92 noiseless mixed-integer bi-objective test functions. Each benchmark function is provided in dimensions [5, 10, 20, 40, 80, 160] with 15 instances. Please refer to the paper for details about each benchmark function.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use. It must be in the range of [1, 92].
Abstract The blackbox optimization benchmarking largescale (bbob-largescale) test suite comprises high-dimensional 24 noiseless single-objective test functions. Each benchmark function is provided in dimensions [20, 40, 80, 160, 320, 640] with 15 instances. Please refer to the paper for details about each benchmark function.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use. It must be in the range of [1, 24].
Abstract The blackbox optimization benchmarking mixed-integer (bbob-mixint) test suite comprises 24 noiseless mixed-integer single-objective test functions. Each benchmark function is provided in dimensions [5, 10, 20, 40, 80, 160] with 15 instances. Please refer to the paper for details about each benchmark function.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use. It must be in the range of [1, 24].
Abstract The blackbox optimization benchmarking noisy (bbob-noisy) test suite comprises 30 noisy test functions. Each benchmark function is provided in dimensions [2, 3, 5, 10, 20, 40] with 15 instances. Please refer to the paper for details about each benchmark function.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob benchmark function to use. It must be in the range of [101, 130]. dimension: Dimension of the benchmark function.
Abstract This package provides a wrapper of the COCO experiments libarary’s bbob-constrained test suite.
APIs class Problem(function_id: int, dimension: int, instance_id: int = 1) function_id: ID of the bbob-constrained benchmark function to use. It must be in the range of [1, 54]. dimension: Dimension of the benchmark function. It must be in [2, 3, 5, 10, 20, 40]. instance_id: ID of the instance of the benchmark function. It must be in the range of [1, 15].
Abstract This package provides a wrapper of the optproblems library’s DTLZ test suite, which consists of 7 kinds of continuous problems with variadic objectives and variables. For the details of the benchmark problems, please take a look at the original paper (Deb et al., 2001) in the reference section.
APIs class Problem(function_id: int, n_objectives: int, dimension: int, k: int, **kwargs: Any) function_id: Function ID of the WFG problem in [1, 9].
Abstract This package provides a wrapper of the optproblems library’s WFG test suite, which consists of 9 kinds of continuous problems with variadic objectives and variables. For the details of the benchmark problems, please take a look at the original paper (Huband et al., 2006) in the reference section.
APIs class Problem(function_id: int, n_objectives: int, dimension: int, k: int | None = None, **kwargs: Any) function_id: Function ID of the WFG problem in [1, 9].
Abstract This package provides a wrapper of the optproblems library’s ZDT test suite, which consists of 6 kinds of benchmark problems. For the details of the benchmark problems, please take a look at the original paper (Zitzler et al., 2000) in the reference section.
APIs class Problem(function_id: int, **kwargs: Any) function_id: Function ID of the WFG problem in [1, 9]. kwargs: Arbitrary keyword arguments, please refer to the optproblems documentation for more details.