knowledge graph embedding
**Knowledge Graph Embeddings** are the **representation learning techniques that map entities and relations in a knowledge graph into continuous low-dimensional vector spaces** — enabling link prediction (are two entities related?), entity classification, and question answering by learning geometric relationships where relation semantics are encoded as transformations between entity embeddings, achieving efficient knowledge base completion at scales infeasible for symbolic reasoning.
**Knowledge Graph Structure**
- Knowledge graph: Set of triples (head, relation, tail) = (h, r, t).
- Example: (Albert_Einstein, born_in, Ulm), (Ulm, located_in, Germany), (Einstein, award, Nobel_Prize).
- Scale: Freebase: 1.9B triples; Wikidata: 10B+ triples; Google Knowledge Graph: 500M entities.
- Link prediction task: Given (h, r, ?), predict the missing tail entity.
**TransE (Bordes et al., 2013)**
- Model: h + r ≈ t → embedding of h plus relation vector r should point near t.
- Loss: Maximize margin: L = max(0, γ + ||h+r-t||₂ - ||h'+r-t'||₂) over negative triples (h', r, t').
- Elegant and simple → works well for 1-to-1 relations.
- Limitation: Cannot model symmetric (r(a,b) → r(b,a)), one-to-many, many-to-one relations.
**TransR and RotatE**
- **TransR**: Project entity embeddings into relation-specific space before computing h+r≈t.
- Per-relation projection matrix M_r: h_r = hM_r, t_r = tM_r → more expressive.
- **RotatE (Sun et al., 2019)**: Each relation as rotation in complex space.
- t = h ∘ r where ∘ is element-wise complex multiplication, |r| = 1.
- Handles: Symmetry (r² = identity → r = unit rotation of π), anti-symmetry, inversion, composition.
- Complex embeddings: h, r, t ∈ ℂ^d → h ∘ r_φ rotates h by angle φ toward t.
**Bilinear Models: DistMult and ComplEx**
- **DistMult**: score(h,r,t) = h · diag(r) · t^T (Hadamard product of h, r, t, then sum).
- Simple, effective → only handles symmetric relations.
- **ComplEx**: Extends DistMult to complex numbers → handles asymmetric relations.
- score = Re(h · r · t̄) where t̄ is complex conjugate.
- Among simplest models that handle all relation patterns.
**Neural Models: ConvE and TuckER**
- **ConvE**: Concatenate flattened h and r → reshape → 2D convolution → linear → dot with t.
- Models higher-order feature interactions between h and r.
- **TuckER**: Tucker decomposition of 3D binary tensor → W ×₁ h ×₂ r ×₃ t.
- Full expressive power within rank constraint.
**Evaluation Metrics**
| Metric | Definition | Better |
|--------|------------|--------|
| MR (Mean Rank) | Average rank of correct entity | Lower |
| MRR (Mean Reciprocal Rank) | Mean of 1/rank | Higher |
| Hits@1 | % of correct entities ranked #1 | Higher |
| Hits@10 | % of correct in top 10 | Higher |
**Benchmarks**
- FB15K-237 (Freebase subset, 237 relations): Standard link prediction benchmark.
- WN18RR (WordNet subset): Hierarchical relations (hypernymy, meronymy).
- YAGO3-10: Entity attributes + relations.
**Knowledge Graphs + LLMs**
- LLMs encode implicit knowledge but hallucinate facts → KG provides structured, verifiable facts.
- Retrieval: Query KG → retrieve relevant triples → inject into LLM prompt (KGRAG).
- Joint training: KGPT, KEPLER embed KG triples into LLM pretraining.
- GraphRAG (Microsoft): Use KG structure to summarize long documents → better retrieval.
Knowledge graph embeddings are **the geometric distillation of world knowledge that enables efficient reasoning over billions of facts** — by representing entities and relations as points and transformations in vector space, KG embedding methods learn that "Paris is to France as Berlin is to Germany" emerges naturally from the embedding geometry, enabling scalable link prediction and knowledge completion that powers recommendation systems, question answering, and drug-drug interaction prediction without the computational cost of explicit symbolic reasoning over millions of rule chains.