Skip to content
Algorithm Atlas
Machine LearningAll categories
CoreSource checked

Long Short-Term Memory

Uses gates and a memory cell to model longer sequence dependencies.

Explore relationshipsOpen constellationBrowse atlas

Quick Facts

Compact metadata for implementation and discovery.

Category
Machine Learning
Difficulty
Advanced
Family
Neural Network
Tier
Core
Status
Verified
Review
Source Checked
Sources
2
Formulas
1
Relationships
4
Code example
Ready
Visualization
Ready
Aliases
LSTM

Overview

Long Short-Term Memory networks add input, forget, and output gates that control how information enters, remains in, and leaves the cell state.

Core Concepts

memory cellforget gateinput gatesequence modeling

Quick facts

Complexity

Runtime and memory behavior at a glance.

Best

O(T h^2)

Average

O(T h^2)

Worst

O(T h^2)

Space

O(T h)

When to Use

  • Use when training predictive models needs memory cell.
  • Use when the input matches the assumptions behind neural Network.

When Not to Use

  • Avoid when the dataset is too small or poorly labeled for the model capacity being used.
  • Avoid when a simpler algorithm gives the required result with less setup.

What It Does

  • Uses gates and a memory cell to model longer sequence dependencies
  • Uses memory cell and forget gate to guide the work.
  • Produces trained model.

Use Cases

  • Training predictive models
  • Finding patterns in data
  • Building learned representations

Inputs

  • Training data
  • Model parameters
  • Loss or scoring rule

Outputs

  • Trained model
  • Predictions
  • Training diagnostics

Key Formulas (1)

LSTM cell update

ct = ft · c_{t−1} + it · gt
c_t = f_t * c_{t-1} + i_t * g_t

Combines retained memory and candidate memory in an LSTM cell.

Pseudocode

  1. 1.Prepare the inputs and state needed for Long Short-Term Memory.
  2. 2.Apply the memory cell step while the stopping condition has not been met.
  3. 3.Update intermediate state and discard invalid candidates.
  4. 4.Return trained model.

Canonical Example

Input
Long Short-Term Memory with a minimal inspectable input
Output
Trained model
Explanation
Long Short-Term Memory applies memory cell to produce trained model.

Implementation Notes

  • Separate training, validation, and test data before tuning model choices.
  • Keep the memory cell step separate enough to test independently.

Common Pitfalls

  • Evaluating only on training data hides overfitting and data leakage.
  • Forgetting to check edge cases can make Long Short-Term Memory look correct on only ideal inputs.

Interactive Mechanism Walkthrough

Iterative Convergence & State Trajectory

Show data, model state, and the training signal that drives updates.

Long Short-Term Memory mechanism walkthrough step 1 of 4x₀x₁x₂x*
Step 1 of 4: Prepare the inputs and state needed for Long Short-Term Memory.
machine-learning-mechanism

Notes

Long Short-Term Memory should be used with the assumptions and tradeoffs of neural Network in mind.

Sources (2)

What each reference supports for this entry.

Long Short-Term Memory

Paper

Original LSTM paper supporting gated memory cells for recurrent neural networks.

Open source

Deep Learning

Book

Deep learning reference covering Long Short-Term Memory model architecture, objective, gradient flow, or training behavior.

Open source

Explore Next

Good next jumps from the relationship graph, ranked for useful discovery.

Open discoveryOpen map
Variant ofRecurrent Neural Network

Machine Learning · Core

LSTM is a gated recurrent neural network variant.

Compare with this
Alternative toGRU

Machine Learning · Core

LSTM and GRU are both gated alternatives for sequence modeling.

Compare with this

Relationships

View full relationship page
Alternative toGRU

LSTM and GRU are both gated alternatives for sequence modeling.

Variant ofRecurrent Neural Network

LSTM is a gated recurrent neural network variant.

Related Algorithms

Recurrent Neural NetworkGRUBackpropagation

Tags

Machine LearningNeural NetworksSequence ModelingGating