Overview
Levenberg-Marquardt adjusts a damping term so steps behave like Gauss-Newton near the solution and like gradient descent when far away.
SpecializedSource checked
Levenberg-Marquardt
Solves nonlinear least-squares problems by blending Gauss-Newton steps with damping.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Optimization
- Difficulty
- Advanced
- Family
- Damped Least Squares
- Tier
- Specialized
- Status
- Verified
- Review
- Source Checked
- Sources
- 1
- Formulas
- 1
- Relationships
- 2
- Code example
- Ready
- Visualization
- Ready
Core Concepts
least-squares residualJacobiandampingtrust region
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(n^3)
Average
O(k n^3)
Worst
O(k n^3)
Space
O(n^2)
When to Use
- Use when the objective is naturally expressed as nonlinear least squares.
- Use when curve fitting is a central requirement.
When Not to Use
- Avoid when the residual Jacobian is unavailable or the problem is not least-squares shaped.
- Avoid when the objective is noisy, discontinuous, or violates the method's assumptions.
What It Does
- Builds a local least-squares model
- Adds damping for stability
- Updates parameters from residuals
Use Cases
- Curve fitting
- Bundle adjustment
- Nonlinear calibration
Inputs
- Residual function
- Jacobian
- Initial parameters
Outputs
- Fitted parameters
- Residual error
Key Formulas (1)
Damped normal equation
(JT J + λ I) δ = −JT r
(J^T J + lambda I) delta = -J^T rComputes a stabilized parameter update from residuals and the Jacobian.
Pseudocode
- 1.Compute residuals and Jacobian
- 2.Solve a damped normal equation
- 3.Evaluate whether the step improves error
- 4.Decrease damping after good steps
- 5.Increase damping after rejected steps
Canonical Example
- Input
- points follow a curved trend, initial parameters are rough
- Output
- parameters fit the curve more closely
- Explanation
- Damped steps reduce squared residuals without jumping too aggressively.
Implementation Notes
- Update damping based on the actual reduction achieved by the step.
- Track objective value, step size, and convergence criteria at each iteration.
- Keep a tiny example with expected output beside the implementation.
Common Pitfalls
- Using one fixed damping value can be either unstable or unnecessarily slow.
- Stopping too early or using unscaled variables can make a good method look broken.
- Not defining boundary, empty, or degenerate inputs before implementation.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryShow a fitting curve taking cautious damped steps toward lower residual error.
Step 1 of 5: Compute residuals and Jacobian
optimization-landscapeNotes
It is designed for sums of squared residuals, not arbitrary objectives.
Sources (1)
What each reference supports for this entry.
Marquardt 1963
PaperFoundational paper for the Levenberg-Marquardt strategy for nonlinear least squares.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
Relationships
View full relationship pageAlternative toBFGS
Both use curvature-like information but target different nonlinear optimization settings.
SpecializesNewton's Method
Levenberg-Marquardt is a damped Newton-style method for nonlinear least squares.
Related Algorithms
Tags
OptimizationTrust RegionGradient BasedRegression