Weight sharing uses the same parameters across multiple parts of a model, reducing parameter count significantly. Applications: Tied embeddings: Input and output embeddings share weights. Common in language models. Reduces parameters by vocabulary_size x hidden_dim. Layer sharing: Same layer weights used at multiple depths (ALBERT). Reduces params proportional to sharing factor. Convolutional: CNNs inherently share weights across spatial positions. Core idea enabling efficient image processing. Universal transformers: Share transformer layer weights across all depths. Benefits: Fewer parameters, regularization effect (constraints model), smaller storage. Trade-offs: May limit capacity, same computation as unshared (in inference). Memory savings primarily in weight storage. ALBERT analysis: 18x fewer parameters than BERT-large with similar performance through aggressive sharing. Tied embeddings specifically: Very common, virtually free improvement. Language models almost always tie input/output embeddings. Implementation: Simply use same nn.Parameter object in multiple places. Gradients accumulate from all uses. When to use: Parameter-constrained settings, when similar computation appropriate at multiple locations.