Overview
Logistic Regression computes a weighted linear score and maps it through a sigmoid or softmax function to estimate class probabilities.
FoundationSource checked
Logistic Regression
Models class probability by applying a logistic function to a linear score.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Machine Learning
- Difficulty
- Beginner
- Family
- Supervised Learning
- Tier
- Foundation
- Status
- Verified
- Review
- Source Checked
- Sources
- 1
- Formulas
- 1
- Relationships
- 5
- Code example
- Not ready
- Visualization
- Ready
Core Concepts
linear scoresigmoidlog lossdecision boundary
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(nd)
Average
O(ndt)
Worst
O(ndt)
Space
O(d)
When to Use
- Use it as a strong, interpretable baseline for classification.
- Use it when calibrated probabilities or linear decision boundaries are acceptable.
- Use regularization for sparse or high-dimensional features.
When Not to Use
- Avoid it when nonlinear feature interactions dominate and features are not engineered for them.
- Avoid interpreting coefficients without considering scaling and correlation.
- Avoid it when classes are extremely imbalanced without calibration or weighting.
What It Does
- Estimates class probabilities
- Learns a decision boundary
Use Cases
- Binary classification
- Risk scoring
- Simple interpretable classifiers
Inputs
- Feature matrix
- Class labels
Outputs
- Learned weights
- Class probabilities
- Predicted labels
Key Formulas (1)
Sigmoid probability
p = 1 / (1 + exp(−z))
p = 1 / (1 + exp(-z))Maps a linear score into a probability-like value between 0 and 1.
Pseudocode
- 1.Initialize model weights
- 2.Compute a weighted score for each example
- 3.Convert scores to probabilities
- 4.Measure classification loss
- 5.Update weights to reduce the loss
Canonical Example
- Input
- examples = [[size, price], ...], target = label or value
- Output
- model predictions for new examples
- Explanation
- Logistic Regression: The algorithm learns a pattern from examples and applies it to unseen data.
Implementation Notes
- Use a numerically stable loss implementation for large positive or negative scores.
- Scale features when using gradient-based solvers.
- Choose binary sigmoid or multiclass softmax based on the label structure.
Common Pitfalls
- Forgetting regularization on high-dimensional features.
- Using accuracy alone on imbalanced data.
- Treating output scores as calibrated probabilities without checking calibration.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryShow a boundary separating two groups of points.
Step 1 of 5: Initialize model weights
decision-boundaryNotes
Despite its name, logistic regression is usually used for classification.
Sources (1)
What each reference supports for this entry.
The Elements of Statistical Learning
BookStatistical learning reference for regression, classification, clustering, trees, and ensemble methods.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
AlternativesNaive Bayes
Machine Learning · Core
Naive Bayes and logistic regression are simple probabilistic classifiers with different independence assumptions.
Compare with thisRelationships
View full relationship pageGeneralizesSoftmax Classifier
Softmax classification extends logistic-style classification to multiple classes.
Alternative toSupport Vector Machine
Both solve classification problems with different loss and margin assumptions.
Related Algorithms
Tags
Machine LearningSupervised LearningClassificationOptimization