« Back to top page

Information Gain

Information Gain

Max-value Entropy Search Sampler

Class or Function Names MESSampler Installation pip install scipy torch Overview Optuna’s acquisition functions all score a candidate by improvement over the incumbent (LogEI, LogPI) or by a posterior quantile (UCB, LCB). Max-value entropy search scores it instead by information: how much observing f(x) is expected to reduce the entropy of the distribution of the maximum value y* = max_x f(x). Writing mu(x), sigma(x) for the posterior mean and standard deviation, phi and Psi for the standard normal PDF and CDF, and gamma = (y* - mu(x)) / sigma(x), the acquisition is