Overview
A Support Vector Machine finds a boundary that separates labeled examples with the largest possible margin, allowing slack for imperfect separation. Kernel functions can make the boundary nonlinear in the original feature space.
CoreSource checked
Support Vector Machine
Learns a separating decision boundary by maximizing the margin around support vectors.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Machine Learning
- Difficulty
- Intermediate
- Family
- Supervised Learning
- Tier
- Core
- Status
- Verified
- Review
- Source Checked
- Sources
- 1
- Formulas
- 1
- Relationships
- 4
- Code example
- Not ready
- Visualization
- Ready
- Aliases
- SVM
Core Concepts
learned parametersclass decision boundarylabeled examplesobjective functionsupervised learningmodel fittraining examplesprediction rule
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(n d)
Average
O(n^2 d)
Worst
O(n^3)
Space
O(n^2)
When to Use
- Use when a margin-based classifier fits the data scale and feature representation.
- Use kernels when nonlinear boundaries are useful and training cost is acceptable.
When Not to Use
- Avoid kernel SVMs on very large datasets without approximation.
- Avoid when feature scaling and validation cannot be handled carefully.
What It Does
- Optimizes a margin-based classification objective
- Identifies support vectors that define the boundary
- Uses kernels when nonlinear separation is needed
Use Cases
- Classification on medium-sized feature sets
- High-dimensional sparse features
- Margin-based baselines with clear regularization
Inputs
- Feature matrix
- Class labels
- Kernel choice
Outputs
- Decision boundary
- Predicted labels
Key Formulas (1)
Margin score
score = dot(weights, features) + bias
score = dot(weights, features) + biasClassifies points by the side and distance from the separating hyperplane.
Pseudocode
- 1.Prepare scaled training features and labels
- 2.Choose linear or kernel representation
- 3.Optimize the margin objective with regularization
- 4.Store support vectors and model coefficients
- 5.Predict by evaluating the signed decision function
Canonical Example
- Input
- two labeled 2D point groups separated by a gap
- Output
- a separating line with maximum margin
- Explanation
- Only the closest points to the boundary become support vectors that determine the margin.
Implementation Notes
- Scale features using training data only.
- Tune regularization and kernel parameters with validation.
- Inspect support-vector count as a rough complexity diagnostic.
Common Pitfalls
- Unscaled features can dominate the margin.
- Kernel and regularization parameters can overfit badly.
- Accuracy alone may hide poor calibration or class-imbalance behavior.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryShow a boundary with margin lines touching support vectors.
Step 1 of 5: Prepare scaled training features and labels
maximum-marginNotes
SVMs can be powerful, but feature scaling and parameter validation are part of the method in practice.
Sources (1)
What each reference supports for this entry.
The Elements of Statistical Learning
BookStatistical learning reference for the model family and tradeoffs of Support Vector Machine.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
Alternative toLogistic Regression
Machine Learning · Foundation
SVMs optimize a margin objective, while logistic regression optimizes log loss.
Compare with thisRelationships
View full relationship pageAlternative toLogistic Regression
SVMs optimize a margin objective, while logistic regression optimizes log loss.
Alternative toSoftmax Classifier
SVMs use margin-based decisions, while softmax classifiers optimize multiclass probability scores.
Related Algorithms
Tags
Machine LearningClassificationSupervised LearningOptimization