Knowledge editing updates a model's stored factual knowledge without expensive full retraining. Why needed: Facts change (new president, updated statistics), training data had errors, personalization requirements. Knowledge storage hypothesis: MLPs in middle-late layers store key-value factual associations. Editing targets these parameters. Methods: ROME (Rank-One Model Editing): Identify layer storing fact, compute rank-one update to change association. MEMIT: Extends ROME to batch edit thousands of facts. MEND: Meta-learned editor network. Locate-then-edit: First find responsible neurons, then update. Edit specification: State change as (subject, relation, old_object → new_object). Model should answer queries about subject with new object. Challenges: Generalization: Handle paraphrases of the query. Locality: Don't break other knowledge. Coherence: Related knowledge stays consistent. Scalability: Many edits accumulate issues. Evaluation benchmarks: CounterFact, zsRE. Comparison to RAG: RAG keeps knowledge external (easier updates), editing modifies model (no retrieval latency). Limitation: Only works for factual knowledge, not complex reasoning or skills.
Related Topics
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.