Random Search is a hyperparameter optimization method that samples random combinations from specified distributions — proven by Bergstra & Bengio (2012) to be more efficient than Grid Search for most ML problems because it explores more unique values of important hyperparameters, can be stopped at any time with a "good enough" result, is embarrassingly parallel (every trial is independent), and requires no assumptions about the objective function landscape.

What Is Random Search?

• Definition: A hyperparameter tuning strategy that randomly samples configurations from the search space (e.g., learning rate drawn from log-uniform [1e-5, 1e-1], batch size drawn from {16, 32, 64, 128}) and evaluates each combination independently, keeping the best result.
• The Key Insight: In high-dimensional hyperparameter spaces, not all hyperparameters matter equally. Learning rate might determine 80% of performance while weight decay matters only 5%. Grid Search wastes time testing many weight decay values while keeping learning rate fixed. Random Search samples more unique learning rate values per trial.
• Why It Beats Grid Search: For a grid of 9 points (3×3), Grid Search tries only 3 unique values per dimension. Random Search with 9 points tries 9 unique values per dimension — 3× more exploration of each axis.

Random Search vs Grid Search (Visual Explanation)

If learning rate is the important parameter, Random Search explores 3× more of its range.

When to Use Which

Python Implementation

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import loguniform, randint

param_distributions = {
    'learning_rate': loguniform(1e-4, 1e-1),
    'max_depth': randint(3, 12),
    'n_estimators': randint(100, 1000),
    'subsample': [0.6, 0.7, 0.8, 0.9, 1.0],
}

search = RandomizedSearchCV(
    model, param_distributions,
    n_iter=100, cv=5, scoring='accuracy',
    random_state=42, n_jobs=-1
)
search.fit(X_train, y_train)
print(f"Best: {search.best_params_}")

Practical Guidelines

Random Search is the practical default for hyperparameter optimization — providing better coverage of important hyperparameters than Grid Search at the same computational budget, supporting any distribution (continuous, discrete, categorical), and offering the unique advantages of being embarrassingly parallel (run on 100 GPUs simultaneously) and anytime-stoppable (the best result so far is always valid).