knowledge graph

**Knowledge Graphs** are the **structured representations of real-world entities and their relationships as a labeled directed graph — nodes represent entities, edges represent semantic relationships — enabling logical reasoning, inference, and factual grounding that pure neural models cannot achieve reliably** — powering search engines, recommendation systems, scientific discovery platforms, and retrieval-augmented AI applications. **What Is a Knowledge Graph?** - **Definition**: A graph-based knowledge base where nodes represent entities (people, organizations, concepts, events) and edges represent typed semantic relationships between them, expressed as (Subject, Predicate, Object) triples. - **Example Triples**: (TSMC, headquartered_in, Hsinchu), (TSMC, manufactures, N3_process_node), (N3_process_node, transistor_density, "300M per mm²"). - **Ontology**: Defines the schema — entity types (Person, Organization, Technology) and valid relationship types (founded_by, subsidiary_of, manufactures) — ensuring semantic consistency. - **Scale**: Wikidata: 100M+ triples; Google Knowledge Graph: 500B+ facts; pharmaceutical knowledge graphs: billions of molecular interaction triples. **Why Knowledge Graphs Matter** - **Factual Grounding**: Ground LLM responses in verified structured facts — dramatically reducing hallucinations on entity-centric questions about people, companies, and events. - **Multi-Hop Reasoning**: Answer complex queries requiring traversal of multiple relationships: "Find all companies that supply components to TSMC's top customers" — impossible with pure text retrieval. - **Semantic Search Enhancement**: Google, Bing, and DuckDuckGo use knowledge graphs to show entity panels, answer direct questions, and disambiguate search intent. - **Drug Discovery**: Biomedical knowledge graphs connecting genes, proteins, diseases, drugs, and pathways enable hypothesis generation for drug-target identification at scale. - **Recommendation Systems**: Entity-relationship graphs power explainable recommendations — "Recommended because you watched films by director X who also directed Y." **Knowledge Graph Construction** **Manual Curation**: - Expert curators define schema and populate facts. Highest accuracy; extremely expensive and slow. - Wikidata: community-curated with 24,000+ contributors; the largest open-source general KG. **Automated Extraction (Information Extraction)**: - Named Entity Recognition → Relation Extraction → Entity Linking → Knowledge Graph population. - Processes millions of documents automatically; lower precision than curation, requires quality filtering. **Distant Supervision**: - Use existing KG facts as weak labels — if KG says (TSMC, headquartered_in, Hsinchu), any sentence mentioning both might express that relation. - Noisy but scales to large corpora without hand-labeled data. **LLM-Based Construction**: - Prompt GPT-4 or Claude to extract (Subject, Predicate, Object) triples from documents in structured JSON. - State-of-the-art quality; costs scale with document volume. **Semantic Web Standards** - **RDF (Resource Description Framework)**: Standard data model for KG triples; URIs identify entities and predicates unambiguously across systems. - **OWL (Web Ontology Language)**: Defines entity class hierarchies, property constraints, and inference rules. Enables logical reasoning over KG facts. - **SPARQL**: Query language for RDF knowledge graphs — the SQL equivalent for structured knowledge retrieval. - **Property Graphs**: Alternative representation (Neo4j) with richer edge properties; more practical for engineering teams than RDF. **Knowledge Graph + LLM Integration** **GraphRAG (Graph Retrieval-Augmented Generation)**: - Microsoft's approach: extract KG from document corpus, retrieve entity neighborhoods for complex queries. - Multi-hop retrieval via graph traversal outperforms vector search on relationship-heavy questions. **KG-Augmented Generation**: - Retrieve relevant KG subgraphs as structured context for LLM prompts. - Reduces hallucination on factual claims about entities with verifiable structured attributes. **LLM for KG Completion**: - Use LLMs to predict missing KG facts (link prediction): "What is the likely relationship between Drug X and Protein Y?" - Combines neural language understanding with structured graph constraints. **Graph Neural Networks for KG Reasoning**: - R-GCN, CompGCN, RotatE: Learn entity and relation embeddings capturing KG structure for link prediction and question answering. **Major Knowledge Graphs** | KG | Domain | Scale | Access | |----|--------|-------|--------| | Wikidata | General | 100M+ triples | Open (SPARQL) | | Google Knowledge Graph | General | 500B+ facts | API | | Freebase | General | 3B triples | Archived | | UniProt / STRING | Biomedical | Billions | Open | | DrugBank | Pharmaceutical | Millions | Open/Commercial | | OpenKG | Chinese | Billions | Open | **Tools & Platforms** - **Neo4j**: Leading property graph database with Cypher query language. Extensive enterprise KG tooling. - **Amazon Neptune**: Managed RDF and property graph database for cloud KG applications. - **Apache Jena**: Open-source Java framework for RDF, OWL, and SPARQL. - **Weaviate / Stardog**: Hybrid vector + knowledge graph systems for semantic search and QA. Knowledge graphs are **the structured memory that transforms AI systems from pattern matchers into systems capable of reliable factual reasoning** — as LLM-assisted KG construction and GraphRAG retrieval patterns mature, knowledge graph integration will become the standard architecture for enterprise AI systems requiring verifiable, traceable answers.