Overview
A Gated Recurrent Unit combines reset and update gates to control how much prior hidden state is retained at each time step.
CoreSource checked
GRU
Models sequences with gated hidden-state updates using fewer gates than LSTM.
Quick Facts
Compact metadata for implementation and discovery.
- Category
- Machine Learning
- Difficulty
- Advanced
- Family
- Neural Network
- Tier
- Core
- Status
- Verified
- Review
- Source Checked
- Sources
- 2
- Formulas
- 1
- Relationships
- 4
- Code example
- Ready
- Visualization
- Ready
- Aliases
- Gated Recurrent Unit
Core Concepts
update gatereset gatehidden statesequence modeling
Quick facts
Complexity
Runtime and memory behavior at a glance.
Best
O(T h^2)
Average
O(T h^2)
Worst
O(T h^2)
Space
O(T h)
When to Use
- Use when training predictive models needs update gate.
- Use when the input matches the assumptions behind neural Network.
When Not to Use
- Avoid when the dataset is too small or poorly labeled for the model capacity being used.
- Avoid when a simpler algorithm gives the required result with less setup.
What It Does
- Models sequences with gated hidden-state updates using fewer gates than LSTM
- Uses update gate and reset gate to guide the work.
- Produces trained model.
Use Cases
- Training predictive models
- Finding patterns in data
- Building learned representations
Inputs
- Training data
- Model parameters
- Loss or scoring rule
Outputs
- Trained model
- Predictions
- Training diagnostics
Key Formulas (1)
GRU hidden update
ht = (1 − zt) · h_{t−1} + zt · hcandidate
h_t = (1 - z_t) * h_{t-1} + z_t * h_candidateBlends previous hidden state with the candidate hidden state.
Pseudocode
- 1.Prepare the inputs and state needed for GRU.
- 2.Apply the update gate step while the stopping condition has not been met.
- 3.Update intermediate state and discard invalid candidates.
- 4.Return trained model.
Canonical Example
- Input
- GRU with a minimal inspectable input
- Output
- Trained model
- Explanation
- GRU applies update gate to produce trained model.
Implementation Notes
- Separate training, validation, and test data before tuning model choices.
- Keep the update gate step separate enough to test independently.
Common Pitfalls
- Evaluating only on training data hides overfitting and data leakage.
- Forgetting to check edge cases can make GRU look correct on only ideal inputs.
Interactive Mechanism Walkthrough
Iterative Convergence & State TrajectoryShow data, model state, and the training signal that drives updates.
Step 1 of 4: Prepare the inputs and state needed for GRU.
machine-learning-mechanismNotes
GRU should be used with the assumptions and tradeoffs of neural Network in mind.
Sources (2)
What each reference supports for this entry.
Deep Learning
BookDeep learning reference covering GRU model architecture, objective, gradient flow, or training behavior.
Open sourceExplore Next
Good next jumps from the relationship graph, ranked for useful discovery.
Relationships
View full relationship pageAlternative toLong Short-Term Memory
GRU is a simpler gated alternative to LSTM.
Variant ofRecurrent Neural Network
GRU is a gated recurrent neural network variant.
Related Algorithms
Tags
Machine LearningNeural NetworksSequence ModelingGating