Overview
Bayesian optimization fits a probabilistic model to observed evaluations and chooses new points that balance exploration and expected improvement.
CoreSource checked
Bayesian Optimization
Optimizes expensive black-box functions using a surrogate model and acquisition rule.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Optimization
- Difficulty
- Advanced
- Family
- Model-Based Optimization
- Tier
- Core
- Status
- Verified
- Review
- Source Checked
- Sources
- 1
- Formulas
- 1
- Relationships
- 1
- Code example
- Ready
- Visualization
- Ready
Core Concepts
surrogate modelacquisition functionexploration-exploitationblack-box objective
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(n^3)
Average
O(n^3)
Worst
O(n^3)
Space
O(n^2)
When to Use
- Use when tuning model parameters needs surrogate model.
- Use when the input matches the assumptions behind model-Based Optimization.
When Not to Use
- Avoid when the objective is cheap and convex enough for a simpler exact solver.
- Avoid when a simpler algorithm gives the required result with less setup.
What It Does
- Optimizes expensive black-box functions using a surrogate model and acquisition rule
- Uses surrogate model and acquisition function to guide the work.
- Produces best candidate.
Use Cases
- Tuning model parameters
- Searching design spaces
- Fitting models to noisy data
Inputs
- Objective function
- Initial candidate or population
- Stopping rule
Outputs
- Best candidate
- Objective value
- Search history
Key Formulas (1)
Expected improvement
EI(x) = E[max(0, best − f(x))]
EI(x) = E[max(0, best - f(x))]Scores a candidate by expected improvement over the current best value.
Pseudocode
- 1.Prepare the inputs and state needed for Bayesian Optimization.
- 2.Apply the surrogate model step while the stopping condition has not been met.
- 3.Update intermediate state and discard invalid candidates.
- 4.Return best candidate.
Canonical Example
- Input
- Bayesian Optimization input with a small representative case
- Output
- Best candidate
- Explanation
- Bayesian Optimization applies surrogate model to produce best candidate.
Implementation Notes
- Track both best-so-far and current candidate state.
- Keep the surrogate model step separate enough to test independently.
Common Pitfalls
- Stopping too early can make a poor local result look like convergence.
- Forgetting to check edge cases can make Bayesian Optimization look correct on only ideal inputs.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryPlot candidate points, objective values, and improvement steps across iterations.
Step 1 of 4: Prepare the inputs and state needed for Bayesian Optimization.
optimization-mechanismNotes
Bayesian Optimization should be used with the assumptions and tradeoffs of model-Based Optimization in mind.
Sources (1)
What each reference supports for this entry.
Shahriari et al. 2016
PaperSurvey reference for Bayesian optimization and surrogate-based global optimization.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
Relationships
View full relationship pageAlternative toGenetic Algorithm
Both can optimize expensive black-box functions.
Related Algorithms
Tags
OptimizationModel FittingStochasticBayesian Optimization