transe

**TransE** (Translating Embeddings for Modeling Multi-Relational Data) is the **foundational knowledge graph embedding model that interprets relations as translation operations in embedding space** — if (head entity h, relation r, tail entity t) is a true fact, then the embedding of h translated by r should approximate the embedding of t, creating a geometric model of symbolic logic that launched the field of neural knowledge graph reasoning. **What Is TransE?** - **Core Idea**: Represent each entity and relation as a vector in the same d-dimensional space. For every true triple (h, r, t), enforce h + r ≈ t — the head entity plus the relation vector should land near the tail entity. - **Score Function**: Score(h, r, t) = -||h + r - t|| — lower distance means higher likelihood of the triple being true. - **Training**: Minimize margin-based loss — true triples must score higher than corrupted triples (random entity substitution) by a fixed margin. - **Bordes et al. (2013)**: The landmark paper that introduced TransE, demonstrating that simple geometric constraints could predict missing facts in Freebase and WordNet with state-of-the-art accuracy. - **Complexity**: O(N × d) parameters — one d-dimensional vector per entity and per relation — extremely parameter-efficient. **Why TransE Matters** - **Simplicity**: Single geometric constraint (translation) captures surprisingly rich relational semantics — relations like "capital of," "directed by," and "is a" all behave as translations. - **Analogy with Word2Vec**: TransE extends the word analogy property (king - man + woman = queen) to multi-relational graphs — entity arithmetic captures factual relationships. - **Speed**: Simple dot products and L2 distances enable fast training on millions of triples — practical for large knowledge bases. - **Foundation**: Every subsequent KGE model (TransR, DistMult, RotatE) either extends or addresses limitations of TransE — it defined the design space. - **Interpretability**: Relation vectors encode semantic directions — "IsCapitalOf" vector consistently points from cities to countries across all training examples. **TransE Strengths and Limitations** **What TransE Models Well**: - **1-to-1 Relations**: Each entity maps to exactly one tail — "capital of" maps each country to exactly one city. - **Simple Hierarchies**: "IsA" and "SubclassOf" relations where direction is consistent. - **Functional Relations**: Relations where the head uniquely determines the tail. **TransE Failure Modes**: - **1-to-N Relations**: "HasChild" — one parent has multiple children. TransE forces all children to have the same embedding (h + r must equal multiple different vectors simultaneously). - **N-to-1 Relations**: "BornIn" — multiple people born in same city. Forces all people to be at same position. - **Symmetric Relations**: "MarriedTo" — if h + r = t then t + r ≠ h unless r = 0. - **Reflexive Relations**: "SimilarTo" — h + r = h implies r = 0 (zero vector), making all reflexive relations identical. **TransE Variants** - **TransH**: Projects entities onto relation-specific hyperplanes — entities have different representations in different relation contexts, handling 1-to-N relations better. - **TransR**: Entities projected into relation-specific entity spaces — explicit mapping between entity and relation spaces. - **TransD**: Dynamic projection matrices derived from both entity and relation vectors — more expressive than TransR with fewer parameters. - **STransE**: Combines TransE with two projection matrices — unifies aspects of TransE and TransR. **TransE Benchmark Results** | Dataset | MR | MRR | Hits@10 | |---------|-----|-----|---------| | **FB15k** | 243 | - | 47.1% | | **WN18** | 251 | - | 89.2% | | **FB15k-237** | 357 | 0.279 | 44.1% | | **WN18RR** | 3384 | 0.243 | 53.2% | **Implementation** - **PyKEEN**: TransE with automatic hyperparameter search, loss variants, and filtered evaluation. - **OpenKE**: C++ optimized TransE for large-scale knowledge bases. - **Custom**: Implement in 20 lines with PyTorch — entity/relation embedding tables, L2 score, margin loss. TransE is **the word2vec of knowledge graphs** — a deceptively simple geometric model that revealed that symbolic logical relationships could be captured by vector arithmetic, launching a decade of research into neural-symbolic reasoning.