Data Privacy
Data privacy, in the context of AI, is the problem of keeping information about real individuals from leaking out of a system that touched their data — not just during collection and storage, but through the trained model itself, which turns out to be a much harder guarantee than it sounds. A model is supposed to learn general patterns, not memorize specific people's records, but training doesn't automatically respect that distinction, and several well-documented attack classes exist specifically to extract individual data back out of a model that was never supposed to reveal it. At a high level, where privacy risk actually enters an AI system looks like this:
Training data extraction attacks proved, concretely and repeatedly, that large language models memorize specific training examples verbatim, not just general patterns. Researchers demonstrated in 2021 that GPT-2 could be prompted to output exact memorized text from its training set — including real names, phone numbers, and email addresses that had appeared in the scraped web data it was trained on — simply by finding the right prompt prefix. Later work on larger models found that memorization scales with model size and with how many times a given piece of text appears in training data, meaning rare but sensitive strings (a unique ID number, a specific person's private message that leaked onto the public web before scraping) are disproportionately likely to be memorized and extractable.
Membership inference attacks don't need to extract any actual content to cause real harm — confirming someone's data was used at all can itself be the privacy violation. If a model is trained specifically on medical records from patients with a particular condition, an attacker who can reliably determine whether a specific named individual's data was in that training set has effectively learned that person's diagnosis, even without extracting a single word of their actual medical record. This is exactly why differential privacy, the formal mathematical framework that bounds how much any single training example can influence a model's output, was developed — it provides a provable guarantee against membership inference and extraction, at a real, measurable cost to model accuracy.
Regulation increasingly treats "the model learned from your data" as a data-processing event subject to the same legal protections as storing that data in a database. GDPR's right to erasure creates a genuine technical problem for AI systems: deleting a person's row from a database is straightforward, but "deleting" their influence from an already-trained model's weights has no clean equivalent — machine unlearning (research into efficiently removing a specific training example's influence without full retraining) exists specifically to try to close that gap, and remains an active, unsolved research area rather than a deployed standard practice. Chatbot and AI-assistant providers now routinely publish data-retention policies specifically addressing whether and how long user conversations are used for further model training, a direct response to users correctly treating a conversation with an AI system as personal data being processed, not just a transient text exchange.
Read data privacy in AI as a problem that doesn't end once training finishes: a model's weights are a lossy but real compression of its training data, extraction and membership-inference attacks are proof that compression can be reversed in part, and differential privacy's accuracy cost is the literal, measurable price of making that reversal mathematically improbable rather than merely inconvenient.