CatBoost: Categorical Boosting

Overview CatBoost (by Yandex) is a high-performance gradient boosting library. Its name comes from "Category" + "Boosting". It is famous for handling categorical data (text labels) automatically without preprocessing, and for its "Overtraining" prevention.

Key Features

1. Native Categorical Support Most algorithms (XGBoost) require you to convert text labels ("Red", "Blue") into numbers (One-Hot Encoding) before training.

• CatBoost handles this internally using "Ordered Target Statistics" (Target Encoding), which is often more accurate and saves memory.

2. Symmetric Trees CatBoost builds balanced (symmetric) trees.

• Benefit: Extremely fast inference (prediction) speed, often 8-20x faster than XGBoost.
• Benefit: Less prone to overfitting.

3. Ordered Boosting A specialized technique to reduce prediction shift, solving a common bias problem in traditional gradient boosting.

Usage

from catboost import CatBoostClassifier

**Define data**
X = [["Red", 10], ["Blue", 20]]
y = [0, 1]
cat_features = [0] # Index of categorical column

**Train**
model = CatBoostClassifier(iterations=100)
model.fit(X, y, cat_features=cat_features)

**Predict**
model.predict([["Red", 15]])

When to use CatBoost?

• You have lots of categorical features (IDs, Cities, User Types).
• You need fast inference in production.
• You want a model that works well with default parameters ("Battle of the defaults").