grid search
**Grid Search** is a **hyperparameter tuning technique that exhaustively evaluates all combinations of specified parameter values** — testing every possibility to find optimal hyperparameters, simple but computationally expensive.
**What Is Grid Search?**
- **Purpose**: Find best hyperparameters for machine learning models.
- **Method**: Test every combination of parameter values.
- **Cost**: Exponential (10 parameters × 5 values = 9.7M combinations).
- **Completeness**: Guaranteed to find best in search space.
- **Speed**: Slow for large spaces, fast for small spaces.
**Why Grid Search Matters**
- **Simple**: Easy to understand and implement.
- **Guaranteed**: Will find best in defined space.
- **Interpretable**: Results show how each parameter affects performance.
- **Baseline**: Good starting point before advanced methods.
- **Parallelizable**: Run combinations simultaneously.
**Grid Search vs Alternatives**
**Grid Search**: Exhaustive, guaranteed optimal, expensive.
**Random Search**: Sample randomly, faster, may miss optimal.
**Bayesian Optimization (Hyperopt)**: Intelligent sampling, 10-100× faster.
**Evolutionary Algorithms**: Population-based, good for large spaces.
**Quick Example**
```python
from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForest
param_grid = {
'n_estimators': [100, 200, 500],
'max_depth': [5, 10, 20],
'min_samples_split': [2, 5, 10]
}
grid = GridSearchCV(
RandomForest(),
param_grid,
cv=5,
n_jobs=-1
)
grid.fit(X_train, y_train)
print(grid.best_params_)
```
**Best Practices**
- Define reasonable parameter ranges first
- Use cross-validation (prevent overfitting)
- Parallelize with n_jobs=-1
- For large spaces, use Random or Bayesian instead
- Use GridSearchCV from sklearn (not manual loops)
Grid Search is the **foundational hyperparameter tuning method** — exhaustive, simple, guaranteed optimal but computationally expensive for large spaces.