Neural Machine Translation (NMT) is the deep learning approach to machine translation that models the probability of a target-language sentence given a source-language sentence using an encoder-decoder neural network — where the transformer architecture with multi-head attention learns to align source and target words without explicit word alignment, achieving translation quality that approaches human parity on high-resource language pairs (English-German, English-Chinese) and enabling multilingual models that translate between 100+ languages with a single model.

Architecture Evolution

Sequence-to-Sequence with Attention (2014-2017):

• Encoder: BiLSTM reads the source sentence and produces a sequence of hidden states.
• Attention: At each decoder step, compute attention weights over encoder states — soft alignment indicates which source words are relevant for generating the current target word.
• Decoder: LSTM generates target words one at a time, conditioned on attention context + previous target word.

Transformer (2017-present):

• Replaces recurrence with self-attention. Encoder: 6-12 layers of multi-head self-attention + feedforward. Decoder: 6-12 layers of masked self-attention + cross-attention to encoder + feedforward.
• Parallelizable (all positions computed simultaneously during training). Scales to much larger models and datasets than RNN-based NMT.
• The dominant NMT architecture by a large margin.

Training

• Data: Parallel corpora — aligned sentence pairs (source, target). WMT datasets: 10-40M sentence pairs per language pair. For low-resource languages: data augmentation (back-translation, paraphrase mining).
• Back-Translation: Train a reverse model (target→source). Translate monolingual target-language text to source language. Use the synthetic parallel data to augment training. Dramatically improves quality — leverages abundant monolingual data.
• Subword Tokenization: BPE (Byte-Pair Encoding) or SentencePiece. Handles rare words by splitting into common subwords. Shared vocabulary between source and target enables cross-lingual sharing.
• Label Smoothing: Replace hard one-hot targets with soft targets (0.9 for correct token, 0.1/V distributed to others). Prevents overconfidence and improves BLEU by 0.5-1.0 points.

Decoding

• Beam Search: Maintain top-K hypotheses at each step (beam size 4-8). Select the highest-scoring complete translation. Without beam search, greedy decoding is 0.5-2.0 BLEU worse.
• Length Normalization: Divide hypothesis score by length^α (α=0.6-1.0) to prevent bias toward short translations.

Multilingual NMT

• Many-to-Many Models: A single model translates between all pairs of N languages. Prepend a target-language tag to the source: "[FR] Hello world" → "Bonjour le monde". Shared vocabulary and shared encoder enable cross-lingual transfer.
• NLLB (No Language Left Behind, Meta): 200 languages, 54B parameters. Specializes with language-specific routing and expert layers. State-of-the-art for low-resource language pairs.
• Zero-Shot Translation: If trained on English↔French and English↔German, the model can translate French↔German (never seen during training) via shared interlingual representations. Quality is lower than direct training but often usable.

Neural Machine Translation is the technology that broke the language barrier at scale — providing the quality and coverage that enables real-time translation of web pages, messages, and documents across hundreds of languages, connecting billions of people who speak different languages.