Precision Machine Learning

Precision, in classification, answers one specific question: of everything the model flagged as positive, how many actually were? It's computed as true positives divided by true positives plus false positives, and the concept dates back further than most machine learning vocabulary — Cyril Cleverdon's Cranfield experiments in the 1960s formalized precision and recall as the standard way to evaluate information retrieval systems, decades before they became standard classifier metrics, and the same quantity appears in medical statistics under the name positive predictive value. A spam filter that flags 100 emails as spam, where 90 really are spam and 10 are legitimate emails caught by mistake, has a precision of 90 divided by 100, or 0.9 — nine times out of ten, trusting the filter's "spam" label is the right call.

Precision only ever looks inside the set of positive predictions — it has nothing to say about what the model missed, which is exactly the job recall covers instead. A model that only flags the single email it's most confident about, and happens to be right, scores a perfect 1.0 precision while catching almost none of the actual spam in the inbox; precision alone can't distinguish a cautious, highly selective model from a comprehensive one, which is why precision is almost never reported in isolation — confusion-matrix lays out the full TP/FP/FN/TN picture these metrics are all computed from, and f1-score covers how precision and recall get combined into one number when both matter.

Moving the model's decision threshold trades precision against recall directly, and the trade is mechanical, not a matter of the model improving or getting worse. Raising the confidence bar required before flagging something positive — say, from 50% confidence to 90% confidence — shrinks the flagged set down to only the clearest cases, which tends to push precision up (fewer borderline false positives sneak through) while recall falls (some real positives that weren't flagged with high confidence get missed entirely); lowering the threshold does the reverse. Plotting precision against recall across every possible threshold produces a precision-recall curve, the standard tool for comparing two models' entire trade-off rather than a single operating point.

Precision at K is the version of this metric that search engines and recommendation systems actually report, because nobody scrolls through every result. Precision@10 asks what fraction of a search engine's top 10 results are actually relevant, which traces directly back to the same Cranfield-era evaluation framework that originally defined precision for ranked retrieval rather than flat classification — a search engine returning 10 results with 8 relevant ones has a precision@10 of 0.8, a number that only looks at the handful of results a real user would actually see rather than every document in the entire corpus.

Precision is the metric to reach for specifically when a false positive is the expensive mistake — flagging a legitimate transaction as fraud and blocking a real customer, or deleting a real email as spam — and it says nothing at all about the opposite kind of mistake, which is exactly why it's always worth asking, before trusting a reported precision number, what that same model's recall looked like on the cases it never flagged in the first place.