CoreSource checked

Lasso Regression

Fits a linear model while encouraging some weights to become exactly zero.

Explore relationships Open constellationBrowse atlas

Quick Facts

Compact metadata for implementation and discovery.

Category: Machine Learning
Difficulty: Intermediate
Family: Regularized Linear Model
Tier: Core
Status: Verified
Review: Source Checked
Sources: 1
Formulas: 1
Relationships: 8
Code example: Ready
Visualization: Ready

Overview

Lasso Regression adds an L1 penalty to squared error, which can perform feature selection by shrinking weak coefficients to zero.

Core Concepts

linear modelL1 penaltysparsityfeature selection

Quick facts

Complexity

Runtime and memory behavior at a glance.

Best

O(n d)

Average

O(k n d)

Worst

O(k n d)

Space

O(d)

When to Use

Use when a sparse linear model or rough feature selection is useful.
Use when feature selection is a central requirement.

When Not to Use

Avoid when many correlated features should share weight rather than compete.
Avoid when the data size, labels, or feature assumptions do not match the model.

What It Does

Computes regularized linear model within Regularized Linear Model workloads (O(k n d) time, O(d) space).
Works by shrinking weak coefficients to zero.
Maintains key invariants across linear model, L1 penalty, sparsity.

Use Cases

Feature selection
Sparse linear models
High-dimensional regression

Inputs

Feature matrix
Target values
Regularization strength

Outputs

Sparse linear weights
Predictions

Key Formulas (1)

Lasso objective

loss = mean((y − Xw)²) + λ · ∑(|w_i|)

loss = mean((y - Xw)^2) + lambda * sum(abs(w_i))

Adds an L1 penalty that can produce exact zero coefficients.

Pseudocode

1.Scale features
2.Choose an L1 regularization strength
3.Optimize squared error with an L1 penalty
4.Use nonzero weights as selected features

Canonical Example

Input: features = 100 measurements, target = score
Output: nonzero weights = selected useful measurements
Explanation: The L1 penalty drives many weak feature weights to zero.

Implementation Notes

Coordinate descent or proximal gradient are common implementation choices.
Split data for validation and keep preprocessing identical between training and inference.
Keep a tiny example with expected output beside the implementation.

Common Pitfalls

Without feature scaling, the penalty can select features for scale rather than usefulness.
Evaluating only on training data can hide overfitting and poor generalization.
Not defining boundary, empty, or degenerate inputs before implementation.

Interactive Mechanism Walkthrough

Iterative Convergence & State Trajectory

Show coefficients shrinking, with small ones reaching zero as lambda increases.

Step 1 of 4: Scale features

model-training

Notes

Lasso Regression operates in O(k n d) time and O(d) space within the Regularized Linear Model family. Key governing relation: loss = mean((y - Xw)^2) + lambda * sum(abs(w_i)). Often compared with Linear Regression and Ridge Regression.