CoreSource checked

Nelder-Mead

Optimizes without derivatives by moving a simplex through the search space.

Explore relationships Open constellationBrowse atlas

Quick Facts

Compact metadata for implementation and discovery.

Category: Optimization
Difficulty: Intermediate
Family: Derivative-Free Optimization
Tier: Core
Status: Verified
Review: Source Checked
Sources: 1
Formulas: 1
Relationships: 2
Code example: Not ready
Visualization: Ready
Aliases: Downhill Simplex

Overview

Nelder-Mead keeps a set of points and changes their shape with reflection, expansion, contraction, and shrink steps.

Core Concepts

objective functionfunction samplingneighbor movederivative-free optimizationiterative improvementstopping rule

Quick facts

Complexity

Runtime and memory behavior at a glance.

Best

O(p)

Average

O(t p)

Worst

O(t p)

Space

O(p^2)

When to Use

Use Nelder-Mead when you need to maintain a simplex of candidate points.
Use it for derivative-free tuning with inputs like objective function.
Use it when you can define a score, objective, constraint, or feasibility test clearly.
Use it when the Derivative-Free Optimization tradeoff is acceptable for the problem size.

When Not to Use

Avoid it when reliable gradients are available and the search space is high-dimensional.
Avoid it when the objective is noisy or changing and the algorithm assumes stable feedback.
Avoid it when the required output is not best point found.

What It Does

Maintains a simplex of candidate points
Moves the simplex based on objective values

Use Cases

Derivative-free tuning
Black-box optimization
Small-dimensional search

Inputs

Objective function
Initial simplex or starting point

Outputs

Best point found

Key Formulas (1)

Reflection point

x_reflected = centroid + α · (centroid − x_worst)

x_reflected = centroid + alpha * (centroid - x_worst)

Moves the worst simplex point through the centroid to explore a better region.

Pseudocode

1.Evaluate all simplex points
2.Reflect the worst point through the centroid
3.Expand if reflection is very good
4.Contract or shrink if reflection is poor
5.Repeat until the simplex is small

Canonical Example

Input: objective = minimize (x - 3)^2, start = x = 0
Output: x moves toward 3
Explanation: Nelder-Mead: The algorithm repeatedly improves a candidate solution according to the objective.

Implementation Notes

Keep the input contract explicit: Objective function; Initial simplex or starting point.
Use this step as the implementation spine: Evaluate all simplex points.
Then preserve the next state change: Reflect the worst point through the centroid.
Log objective values across iterations so regressions and stalled progress are visible.

Common Pitfalls

Nelder-Mead can work well in low dimensions but has weak guarantees.
Stopping after one good-looking result without checking convergence or constraints.
Returning best point found without handling empty, invalid, or boundary inputs.

Interactive Mechanism Walkthrough

Iterative Convergence & State Trajectory

Show a simplex changing shape as it searches a surface.

Step 1 of 5: Evaluate all simplex points

moving-simplex

Notes

Nelder-Mead operates in O(t p) time and O(p^2) space within the Derivative-Free Optimization family. Key governing relation: x_reflected = centroid + alpha * (centroid - x_worst). Often compared with Simulated Annealing and Coordinate Descent.