Abstract HierarchicalTPESampler is a mixture-of-experts variant of Optuna’s multivariate, group-decomposed TPESampler for conditional search spaces (objectives where different trials request different parameters, e.g. if optimizer == "adam": ...).
Why this exists. On a conditional (dynamic) search space, Optuna’s own multivariate TPESampler is inadequate. With group=False the conditional parameters fall back to independent (random) sampling, because a dynamic search space is not supported for multivariate=True. With group=True the space is decomposed into parameter groups that are each sampled independently, so correlation between groups is lost.