Abstract As the Optuna CMA-ES sampler does not support any flexible ways to initialize the parameters of the Gaussian distribution, so I created a workaround to do so.
Class or Function Names UserPriorCmaEsSampler In principle, most arguments follow optuna.samplers.CmaEsSampler, but some parts are modified.
For example, UserPriorCmaEsSampler does not support source_trials and use_separable_cma due to their incompatibility. Instead, we replaced x0 and sigma0 in CmaEsSampler with mu0 and cov0. In CmaEsSampler, we needed to provide x0 as dict and sigma0 only as float.
Abstract This package provides a meta-learning extension of the Tree-structured Parzen Estimator (TPE) that leverages previously completed Optuna studies on related tasks to accelerate optimization.
The algorithm is based on the approach described in:
Speeding Up Multi-Objective Hyperparameter Optimization by Task Similarity-Based Meta-Learning for the Tree-Structured Parzen Estimator (IJCAI 2023) The key idea is to compute task similarity between the target task and source tasks based on the overlap of their promising regions, and then use a weighted mixture of TPE models to guide the search.