Generative Adversarial Networks (GANs) are the class of deep generative models consisting of two competing neural networks — a generator that synthesizes realistic data from random noise and a discriminator that distinguishes generated from real data — trained adversarially until the generator produces outputs indistinguishable from real data.

GAN Architecture:

• Generator (G): maps random noise vector z ~ N(0,1) to data space — typically uses transposed convolutions (ConvTranspose2d) to progressively upsample from low-dimensional noise to full-resolution images
• Discriminator (D): binary classifier distinguishing real from generated samples — typically uses strided convolutions to progressively downsample images to a real/fake probability; architecture mirrors generator in reverse
• Adversarial Training: G minimizes log(1 - D(G(z))) while D maximizes log(D(x)) + log(1 - D(G(z))) — this minimax game converges (theoretically) when G's output distribution matches the real data distribution and D outputs 0.5 for all inputs
• Training Dynamics: alternating updates — train D for k steps (typically k=1) on real and fake batches, then train G for 1 step using D's feedback; delicate balance required to prevent one network from overpowering the other

Training Challenges and Solutions:

• Mode Collapse: generator produces limited diversity, covering only a few modes of the data distribution — solutions: minibatch discrimination, unrolled GAN training, diversity-promoting regularization, or Wasserstein distance
• Training Instability: loss oscillations, gradient vanishing when D too strong — Wasserstein GAN (WGAN) uses Earth Mover's distance with gradient penalty, providing smooth gradients even when D is confident; spectral normalization constraints stabilize D
• Vanishing Gradients: when D perfectly classifies, G receives near-zero gradients — non-saturating loss reformulation (maximize log D(G(z)) instead of minimize log(1-D(G(z)))) provides stronger gradients early in training
• Evaluation Metrics: Frechet Inception Distance (FID) measures distribution similarity between generated and real images — lower FID indicates better quality/diversity; Inception Score (IS) measures quality and diversity independently

GAN Variants:

• StyleGAN: progressive growing with style-based generator — maps noise through a mapping network to style vectors that modulate each layer via adaptive instance normalization; produces photorealistic faces at 1024×1024 resolution
• Conditional GAN (cGAN): both G and D conditioned on class labels or other information — enables controlled generation (e.g., generate images of specific classes); pix2pix uses paired image-to-image translation
• CycleGAN: unpaired image-to-image translation using cycle consistency loss — learns bidirectional mappings (horse↔zebra) without requiring paired training data
• Progressive GAN: training starts at low resolution (4×4) and progressively adds higher-resolution layers — stabilizes training and produces high-quality 1024×1024 images

GANs revolutionized generative modeling by producing the first truly photorealistic synthetic images — while partly superseded by diffusion models for some applications, GANs remain essential for real-time generation, super-resolution, data augmentation, and domain adaptation due to their single-pass inference speed.