distmult
**DistMult** is a **knowledge graph embedding model based on bilinear factorization with diagonal relation matrices** — scoring entity-relation-entity triples by computing the element-wise product of head entity, relation, and tail entity vectors, making it highly effective for symmetric relations while being parameter-efficient and fast to train.
**What Is DistMult?**
- **Definition**: A semantic matching model that scores triples (h, r, t) by the bilinear form: Score(h, r, t) = sum of (h_i × r_i × t_i) over all dimensions — a trilinear dot product of three vectors.
- **Diagonal Simplification**: DistMult simplifies the general bilinear model (RESCAL) by constraining relation matrices to be diagonal — instead of a full d×d matrix per relation, only a d-dimensional vector, dramatically reducing parameters.
- **Yang et al. (2015)**: Introduced DistMult as a simplification of RESCAL that achieves competitive performance with a fraction of the parameters.
- **Symmetry Property**: Score(h, r, t) = Score(t, r, h) by construction — swapping head and tail gives identical score, making DistMult perfectly symmetric.
**Why DistMult Matters**
- **Parameter Efficiency**: O(N × d) parameters for N entities — same as TransE, but the bilinear formulation captures richer interactions than translation.
- **Symmetric Relations**: Naturally models symmetric predicates — "MarriedTo," "SimilarTo," "AlliedWith," "IsColleagueOf" — where the relation holds in both directions.
- **Training Stability**: Trilinear scoring is smooth and differentiable everywhere — no distance calculations or normalization constraints.
- **Strong Baseline**: Despite simplicity, DistMult consistently outperforms TransE on many benchmarks — demonstrates that bilinear models capture relational semantics effectively.
- **Foundation for Complex Models**: ComplEx extends DistMult to complex numbers to handle asymmetry; RotatE extends to rotation — DistMult is the starting point for a major model family.
**DistMult Strengths and Limitations**
**What DistMult Models Well**:
- **Symmetric Relations**: Perfect geometric behavior — h·r·t = t·r·h always.
- **Correlation-Based Relations**: Relations capturing statistical co-occurrence rather than directional causation.
- **Large-Scale KGs**: Parameter efficiency enables training on knowledge graphs with millions of entities.
**DistMult Failure Modes**:
- **Asymmetric Relations**: "FatherOf" cannot be distinguished from "SonOf" — if DistMult learns (Luke, FatherOf, Anakin), it simultaneously predicts (Anakin, FatherOf, Luke) with the same score.
- **Antisymmetric Relations**: "GreaterThan," "LocatedIn" — directional relations where the relationship does not hold when reversed.
- **Composition Patterns**: Cannot easily model relation chains — "BornIn" composed with "LocatedIn" to infer citizenship.
**DistMult vs. Related Models**
| Model | Relation Representation | Symmetric | Antisymmetric | Composition |
|-------|------------------------|-----------|---------------|-------------|
| **DistMult** | Diagonal matrix (vector) | Yes | No | No |
| **RESCAL** | Full matrix | Yes | Yes | Partial |
| **ComplEx** | Complex-valued vector | Yes | Yes | No |
| **RotatE** | Complex rotation | Yes | Yes | Yes |
**DistMult Benchmark Results**
| Dataset | MRR | Hits@1 | Hits@10 |
|---------|-----|--------|---------|
| **FB15k-237** | 0.281 | 0.199 | 0.446 |
| **WN18RR** | 0.430 | 0.390 | 0.490 |
| **FB15k** | 0.654 | 0.546 | 0.824 |
**When to Use DistMult**
- **Symmetric-heavy KGs**: Knowledge graphs dominated by symmetric predicates (social networks, similarity graphs).
- **Rapid Baseline**: DistMult trains in minutes and provides a strong baseline to compare against more complex models.
- **Memory-Constrained**: When ComplEx or RotatE (2x memory for complex numbers) cannot fit in GPU memory.
- **Ensemble Components**: DistMult and ComplEx ensembles often outperform either alone.
**Implementation**
- **PyKEEN**: DistMultModel with automatic negative sampling, filtered evaluation, and early stopping.
- **AmpliGraph**: Built-in DistMult with SGD/Adam optimizers and batch negative sampling.
- **Manual**: 10 lines in PyTorch — entity_emb, rel_emb tables; score = (h * r * t).sum(dim=-1).
DistMult is **symmetric semantic matching** — a beautifully simple bilinear model that captures the correlational structure of knowledge graphs, serving as the essential baseline and foundation for the ComplEx and RotatE model families.