GRU
A gated recurrent unit, or GRU, is a recurrent neural network cell that reads a sequence one step at a time while carrying a hidden state, and that uses two learned gates to decide how much of the old state to keep and how much to overwrite with new information. It was introduced by Kyunghyun Cho, Bart van Merriënboer, Çağlar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio in the 2014 paper "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation", where it served as the building block of an early neural translation model. The motivation was the one that produced the LSTM: plain recurrent networks forget distant context because gradients vanish as they travel back through many steps.
The cell has two gates, and each answers a simple question. The update gate decides how much of the previous hidden state is carried straight forward versus replaced; the reset gate decides how much of the previous state is consulted when proposing a new candidate state. In outline, the candidate is computed from the current input and the reset-scaled old state, and the new state is a blend: new state = (1 − z) × old state + z × candidate, where z is the update gate (some references swap the roles of z and 1 − z). When z is near 0 the old state passes through almost unchanged, creating a near-identity path through time, which is the same remedy for vanishing gradients that the skip connection provides across depth. When the reset gate is near 0 the cell ignores its past and behaves like a fresh feed-forward step, useful at a topic or sentence boundary.
Against the LSTM of Hochreiter and Schmidhuber (1997), the GRU is leaner: it merges the cell state and hidden state into one vector and replaces the forget, input, and output gates with just two. The saving is concrete. With one bias vector per gate, a layer with input size 128 and hidden size 256 needs 3 × (256 × (128 + 256) + 256) = 295,680 parameters as a GRU against 4 × 98,560 = 394,240 as an LSTM, about 25 percent fewer, with a correspondingly lighter and often slightly faster step (frameworks such as PyTorch use two bias vectors, so their counts differ a little). The GRU also exposes no separate output gate, so its full state is visible to the next layer, which makes it a little less able to hide information inside memory.
Which one is better has been tested repeatedly, and the honest result is that neither wins consistently. Chung, Gülçehre, Cho, and Bengio's 2014 "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling" found the two comparable and both clearly ahead of plain tanh recurrence on music and speech data. Jozefowicz, Zaremba, and Sutskever's 2015 exploration of thousands of recurrent architectures found the GRU beating the LSTM on most tasks they tried, but also that an LSTM with its forget-gate bias initialized to 1 closed most of the gap, and Greff and colleagues' 2017 "LSTM: A Search Space Odyssey" found no variant of the LSTM that was reliably better than the standard design. The practical rule is to try both on the task and keep the cheaper one when results tie; the notion that the GRU is merely a weakened LSTM, or that it is categorically superior, are equally unfounded.
Neither gate-based design repaired every limitation of recurrence. Gated cells carry information across tens or a few hundred steps far better than plain recurrent networks, but they still compress an entire history into a fixed-size vector and must process steps strictly in order, which prevents parallelization across time. Attention (Bahdanau, Cho, and Bengio, 2014) and then the transformer (Vaswani and colleagues, 2017) addressed both problems, and large language models moved to attention-based decoders. GRUs remain in use where small size, low latency, and streaming matter: on-device speech and keyword spotting, time-series forecasting and sensor models, small control policies in reinforcement learning, and some compact encoders. Bidirectional and stacked variants, and the cuDNN fused kernels in the major frameworks, make them easy to deploy.
Read the GRU as a recurrent memory trimmed to its essentials: two gates decide what to keep and what to rewrite, delivering most of the LSTM's ability to remember at a smaller cost, while the real lesson from the 2014 comparisons is that gating itself, rather than the exact wiring of the gates, did the heavy lifting.