Regression Analysis

Overview Regression is a type of Supervised Learning where the goal is to predict a continuous numerical value (Temperature, Price, Age), as opposed to a categorical Class (Dog/Cat).

Types

1. Linear Regression Fitting a straight line ($y = mx + b$) to data.

• Metric: R-Squared ($R^2$), Mean Squared Error (MSE).
• Assumptions: Linear relationship, homoscedasticity (constant variance).

2. Polynomial Regression Fitting a curve ($y = ax^2 + bx + c$).

• Risk: Overfitting (wiggling too much to hit every point).

3. Ridge / Lasso Regression Linear regression with Regularization to prevent overfitting.

• L1 (Lasso): Shrinks weights to 0 (Feature Selection).
• L2 (Ridge): Shrinks weights towards 0 (Stability).

Evaluation

• MAE (Mean Absolute Error): "On average, I am off by $5k." (Robust to outliers).
• RMSE (Root Mean Squared Error): "I am off by $5k, but errors are squared." (Penalizes huge errors heavily).

"Regression identifies the relationship between a dependent variable and one or more independent variables."