CoreSource checked

Coordinate Descent

Optimizes one parameter or coordinate at a time.

Explore relationships Open constellationBrowse atlas

Quick Facts

Compact metadata for implementation and discovery.

Category: Optimization
Difficulty: Intermediate
Family: Local Optimization
Tier: Core
Status: Verified
Review: Source Checked
Sources: 1
Formulas: 1
Relationships: 2
Code example: Not ready
Visualization: Ready

Overview

Coordinate Descent repeatedly picks a coordinate, improves the objective along that single direction, and cycles through coordinates.

Core Concepts

objective functioniterative updateneighbor movelocal optimizationiterative improvementstopping rule

Quick facts

Complexity

Runtime and memory behavior at a glance.

Best

O(p)

Average

O(tp)

Worst

O(tp)

Space

O(p)

When to Use

Use Coordinate Descent when you need to choose one coordinate to update.
Use it for sparse regression with inputs like objective function.
Use it when you can define a score, objective, constraint, or feasibility test clearly.
Use it when the Local Optimization tradeoff is acceptable for the problem size.

When Not to Use

Avoid it when the objective is noisy or changing and the algorithm assumes stable feedback.
Avoid it when the required output is not improved parameters.

What It Does

Chooses one coordinate to update
Holds other coordinates fixed during the update

Use Cases

Sparse regression
Separable objectives
Simple constrained optimization

Inputs

Objective function
Initial parameters
Coordinate update rule

Outputs

Improved parameters

Key Formulas (1)

Coordinate update

x_{i next} = argmin_z f(x₁, …, z, …, x_n)

x_i_next = argmin_z f(x_1, ..., z, ..., x_n)

Optimizes one coordinate while holding the others fixed.

Pseudocode

1.Initialize all coordinates
2.Choose the next coordinate
3.Find an improved value for that coordinate
4.Update only that coordinate
5.Repeat until progress is small

Canonical Example

Input: objective = minimize (x - 3)^2, start = x = 0
Output: x moves toward 3
Explanation: Coordinate Descent: The algorithm repeatedly improves a candidate solution according to the objective.

Implementation Notes

Keep the input contract explicit: Objective function; Initial parameters; Coordinate update rule.
Use this step as the implementation spine: Initialize all coordinates.
Then preserve the next state change: Choose the next coordinate.
Log objective values across iterations so regressions and stalled progress are visible.

Common Pitfalls

Coordinate Descent is useful when single-coordinate updates are cheap.
Stopping after one good-looking result without checking convergence or constraints.
Returning improved parameters without handling empty, invalid, or boundary inputs.

Interactive Mechanism Walkthrough

Iterative Convergence & State Trajectory

Show a point moving along one axis at a time.

Step 1 of 5: Initialize all coordinates

axis-steps

Notes

Coordinate Descent operates in O(tp) time and O(p) space within the Local Optimization family. Key governing relation: x_i_next = argmin_z f(x_1, ..., z, ..., x_n). Often compared with Gradient Descent and Nelder-Mead.