ROC Curve & AUC Score

Overview The ROC (Receiver Operating Characteristic) curve and AUC (Area Under the Curve) are performance metrics for binary classification problems specifically at various threshold settings.

The Problem with "Accuracy" If you have 99 "Good" emails and 1 "Spam" email. A model that says "All Good" has 99% accuracy but tells you nothing.

ROC Curve It plots:

• X-axis: False Positive Rate (FPR) - "Crypto scams labeled as legitimate."
• Y-axis: True Positive Rate (TPR/Recall) - "Spam correctly labeled as spam."

As you lower the threshold (e.g., mark it spam if probability > 10% vs > 90%), the TPR goes up, but FPR also goes up. The curve visualizes this trade-off.

AUC (Area Under Curve) A single number summary of the curve (0.0 to 1.0).

• 0.5: Random guessing.
• 1.0: Perfect classifier.
• 0.9: Excellent.

Interpretation "An AUC of 0.8 means there is an 80% chance that the model will rank a random positive instance higher than a random negative instance."

Use AUC when you care about ranking ability, not just the hard label.