Recurrent neural network
A recurrent neural network (RNN) is a species of Machine learning model built to devour sequences one piece at a time, carrying a memory of everything it has seen so far into every new step. Where a plain feedforward network treats each input as an isolated snapshot, an RNN loops a hidden state back into itself — the output of the network at time t becomes part of the input at time t+1. This simple trick of self-reference gives the architecture something like a short attention span for the past, making it a natural fit for text, speech, music, weather readings, or any data where order is meaning.
The loop that remembers
Imagine reading a sentence one word at a time, but forgetting the entire sentence as soon as you finish it — that is a network without recurrence. An RNN instead updates an internal "state" vector at each step, blending new input with a compressed digest of history. Unroll this loop across time and it resembles a very deep feedforward network, one layer per time step, all sharing the same weights. This unrolling is also why RNNs are trained by "backpropagation through time," a variant of ordinary backpropagation that must chain gradients across potentially hundreds of steps.
That chaining is also the architecture's Achilles' heel: gradients can shrink toward zero or explode toward infinity as they travel backward, making early inputs effectively invisible to the network — the notorious vanishing/exploding gradient problem. The most celebrated fix was the Long short-term memory cell, which adds gated pathways letting information skip across time largely untouched, like a bypass lane around the leaky pipe of ordinary recurrence. Gated Recurrent Units offered a simpler cousin of the same idea.
What RNNs built
Before the rise of the Transformer, RNNs were the engine behind Neural machine translation and Sequence to sequence learning, reading a source sentence into a fixed vector and unspooling a translation from it. They powered early Attention mechanisms, speech recognizers, and generative text models, and served as the scaffolding on which ideas like Word embedding and distributed representations (see Word2Vec) were tested at scale. Even the GPT lineage and models like BERT descend intellectually from the sequence-modeling ambitions RNNs first pursued, though they replaced recurrence with self-attention (as argued in Attention Is All You Need) to allow parallel training and longer-range memory.
Beyond text
The recurrent idea generalizes wherever a Coordinate system of time exists: sensor streams, financial series, even simulations in Agent-based modeling where each agent's state evolves step by step. Some researchers have drawn loose analogies between recurrent memory and biological processes in Cells, though the comparison is more poetic than literal. Today RNNs persist in lightweight, low-latency settings and inside Responsive system designs where a User Interface must react to a continuous stream of input.