Encoder-Decoder Architecture

An encoder-decoder architecture is a fundamental design pattern in machine learning and artificial intelligence where a system splits into two complementary components: an encoder that compresses input data into a dense representation, and a decoder that reconstructs or generates output from that representation.

The encoder processes raw input—whether text, images, or audio—and learns to extract its essential features into a compact embedding or context vector. The decoder then takes this distilled information and performs a task: generating translations, creating captions, synthesizing speech, or reconstructing the original input.

This separation mirrors how humans understand information: grasping the essence of something before expressing it differently. The architecture excels at tasks requiring transformation, such as machine translation, image captioning, and conversational AI. By forcing information through a bottleneck, the encoder learns which features matter most, while the decoder learns to work creatively with constraints.

Modern variants—especially Transformer models—have revolutionized the field by allowing encoders and decoders to attend to any part of their input simultaneously, rather than processing sequentially. This architectural choice powers today's most sophisticated language models and scientific applications.

Related

Transformer, Sequence-to-Sequence, Neural Network, Attention Mechanism, Embeddings (machine learning)