Class or Function Names EvoMergeSampler EvoMergeTrial Installation pip install git+https://github.com/arcee-ai/mergekit.git pip install sentencepiece accelerate protobuf bitsandbytes langchain langchain-community datasets pip install pandas cmaes export HF_TOKEN=xxx Example sampler = EvoMergeSampler(base_config="path/to/config/yml/file") study = optuna.create_study(sampler=sampler) for _ in range(100): trial = study.ask() evo_merge_trial = EvoMergeTrial(study, trial._trial_id) model = evo_merge_trial.suggest_model() acc = try_model(model) study.tell(trial, acc) print(study.trials_dataframe(attrs=("number", "value"))) See example.py for a full example. You need GPU with 16G VLAM to run this example. The following figures are obtained from the analysis of the optimization.
Abstract Large Language Models to Enhance Bayesian Optimization (LLAMBO) LLAMBO, by Liu et al., is a novel approach that integrates Large Language Models (LLMs) into the Bayesian Optimization (BO) framework to improve the optimization of complex, expensive-to-evaluate black-box functions. By leveraging the contextual understanding and few-shot learning capabilities of LLMs, LLAMBO enhances multiple facets of the BO pipeline:
Zero-Shot Warmstarting
LLAMBO frames the optimization problem in natural language, allowing the LLM to propose promising initial solutions.