CoreSource checked

Elastic Net

Combines L1 and L2 penalties for sparse but more stable linear models.

Explore relationships Open constellationBrowse atlas

Quick Facts

Compact metadata for implementation and discovery.

Category: Machine Learning
Difficulty: Intermediate
Family: Regularized Linear Model
Tier: Core
Status: Verified
Review: Source Checked
Sources: 1
Formulas: 1
Relationships: 3
Code example: Ready
Visualization: Ready

Overview

Elastic Net blends Lasso and Ridge penalties, making it useful when features are numerous and correlated.

Core Concepts

L1 penaltyL2 penaltyregularization mixlinear model

Quick facts

Complexity

Runtime and memory behavior at a glance.

Best

O(n d)

Average

O(k n d)

Worst

O(k n d)

Space

O(d)

When to Use

Use when you want sparse weights but Lasso is unstable with correlated features.
Use when high-dimensional regression is a central requirement.

When Not to Use

Avoid when a simpler Ridge or Lasso model already meets the need.
Avoid when the data size, labels, or feature assumptions do not match the model.

What It Does

Learns linear weights
Combines L1 and L2 penalties
Balances sparsity and stability

Use Cases

High-dimensional regression
Correlated feature sets
Sparse predictive models

Inputs

Feature matrix
Target values
L1 and L2 strengths

Outputs

Regularized linear weights

Key Formulas (1)

Elastic Net objective

loss = mean((y − Xw)²) + lambda1 · ∑(|w_i|) + lambda2 · ∑(w_i²)

loss = mean((y - Xw)^2) + lambda1 * sum(abs(w_i)) + lambda2 * sum(w_i^2)

Combines sparsity and coefficient shrinkage in one objective.

Pseudocode

1.Standardize features
2.Choose total penalty strength and L1 ratio
3.Optimize squared error plus both penalties
4.Validate the penalty mix

Canonical Example

Input: features = correlated sensor readings
Output: weights = sparse but stable group of predictors
Explanation: Elastic Net keeps regularization from choosing one correlated feature arbitrarily.

Implementation Notes

Cross-validate both the total regularization and the L1 ratio.
Split data for validation and keep preprocessing identical between training and inference.
Keep a tiny example with known output beside the implementation.
Treat examples as educational sketches; production models need validation, monitoring, and domain checks.

Common Pitfalls

Tuning only one penalty parameter can miss the useful sparsity-stability balance.
Evaluating only on training data can hide overfitting and poor generalization.
Not defining boundary, empty, or degenerate inputs before implementation.
Data leakage, unscaled features, and validation on training data can make results look better than they are.

Interactive Mechanism Walkthrough

Iterative Convergence & State Trajectory

Show coefficients shrinking with some reaching zero and correlated groups staying more stable.

Step 1 of 4: Standardize features

model-training

Notes

Elastic Net operates in O(k n d) time and O(d) space within the Regularized Linear Model family. Key governing relation: loss = mean((y - Xw)^2) + lambda1 * sum(abs(w_i)) + lambda2 * sum(w_i^2). Often compared with Ridge Regression and Lasso Regression.