Overview
AdaBoost trains weak learners in sequence, increasing the importance of difficult examples and weighting learners by their accuracy.
CoreSource checked
AdaBoost
Builds a classifier by reweighting examples that previous weak learners misclassified.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Machine Learning
- Difficulty
- Intermediate
- Family
- Ensemble Learning
- Tier
- Core
- Status
- Verified
- Review
- Source Checked
- Sources
- 2
- Formulas
- 1
- Relationships
- 3
- Code example
- Ready
- Visualization
- Ready
Core Concepts
example weightweak learnerweighted voteboosting
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(t n)
Average
O(t n)
Worst
O(t n)
Space
O(n + t)
When to Use
- Use when training predictive models needs example weight.
- Use when the input matches the assumptions behind ensemble Learning.
When Not to Use
- Avoid when the dataset is too small or poorly labeled for the model capacity being used.
- Avoid when a simpler algorithm gives the required result with less setup.
What It Does
- Builds a classifier by reweighting examples that previous weak learners misclassified
- Uses example weight and weak learner to guide the work.
- Produces trained model.
Use Cases
- Training predictive models
- Finding patterns in data
- Building learned representations
Inputs
- Training data
- Model parameters
- Loss or scoring rule
Outputs
- Trained model
- Predictions
- Training diagnostics
Key Formulas (1)
Learner weight
α = 0.5 · ln((1 − error) / error)
alpha = 0.5 * ln((1 - error) / error)Assigns more influence to weak learners with lower weighted error.
Pseudocode
- 1.Prepare the inputs and state needed for AdaBoost.
- 2.Apply the example weight step while the stopping condition has not been met.
- 3.Update intermediate state and discard invalid candidates.
- 4.Return trained model.
Canonical Example
- Input
- AdaBoost with a minimal inspectable input
- Output
- Trained model
- Explanation
- AdaBoost applies example weight to produce trained model.
Implementation Notes
- Separate training, validation, and test data before tuning model choices.
- Keep the example weight step separate enough to test independently.
Common Pitfalls
- Evaluating only on training data hides overfitting and data leakage.
- Forgetting to check edge cases can make AdaBoost look correct on only ideal inputs.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryShow data, model state, and the training signal that drives updates.
Step 1 of 4: Prepare the inputs and state needed for AdaBoost.
machine-learning-mechanismNotes
AdaBoost should be used with the assumptions and tradeoffs of ensemble Learning in mind.
Sources (2)
What each reference supports for this entry.
AdaBoost
PaperOriginal AdaBoost paper supporting adaptive weight updates and weak-learner combination.
Open sourceThe Elements of Statistical Learning
BookStatistical learning reference covering AdaBoost additive model, ensemble behavior, or supervised-learning objective.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
Relationships
View full relationship pageOften paired withDecision Tree
AdaBoost commonly uses shallow decision trees as weak learners.
Alternative toGradient Boosting
AdaBoost and gradient boosting both create sequential boosted ensembles.
Related Algorithms
Tags
Machine LearningEnsembleBoostingClassification