The blackbox optimization benchmarking (bbob) test suite

Abstract

The blackbox optimization benchmarking (bbob) test suite comprises 24 noiseless single-objective test functions. BBOB is one of the most widely used test suites to evaluate and compare the performance of blackbox optimization algorithms. Each benchmark function is provided in dimensions [2, 3, 5, 10, 20, 40] with 110 instances.

APIs

class Problem(function_id: int, dimension: int, instance_id: int = 1)

• function_id: ID of the bbob benchmark function to use. It must be in the range of [1, 24].
• dimension: Dimension of the benchmark function. It must be in [2, 3, 5, 10, 20, 40].
• instance_id: ID of the instance of the benchmark function. It must be in the range of [1, 110].

Methods and Properties

• search_space: Return the search space.
  • Returns: dict[str, optuna.distributions.BaseDistribution]
• directions: Return the optimization directions.
  • Returns: list[optuna.study.StudyDirection]
• __call__(trial: optuna.Trial): Evaluate the objective function and return the objective value.
  • Args:
    • trial: Optuna trial object.
  • Returns: float
• evaluate(params: dict[str, float]): Evaluate the objective function given a dictionary of parameters.
  • Args:
    • params: Decision variable like {"x0": x1_value, "x1": x1_value, ..., "xn": xn_value}. The number of parameters must be equal to dimension.
  • Returns: float

The properties defined by cocoex.Problem are also available such as number_of_objectives.

Installation

Please install the coco-experiment package.

pip install -U coco-experiment

Example

import optuna
import optunahub


bbob = optunahub.load_module("benchmarks/bbob")
sphere2d = bbob.Problem(function_id=1, dimension=2, instance_id=1)

study = optuna.create_study(directions=sphere2d.directions)
study.optimize(sphere2d, n_trials=20)

print(study.best_trial.params, study.best_trial.value)

List of Benchmark Functions

Please refer to the paper for details about each benchmark function.

Category

1. Separable Functions
2. Functions with low or moderate conditioning
3. Functions with high conditioning and unimodal
4. Multi-modal functions with adequate global structure
5. Multi-modal functions with weak global structure

Reference

Finck, S., Hansen, N., Ros, R., & Auger, A. Real-Parameter Black-Box Optimization Benchmarking 2010: Presentation of the Noiseless Functions.