BentoML: Unified Model Serving

Overview BentoML is an open-source framework for building reliable machine learning serving endpoints. It solves the "It works on my notebook" problem by packaging the model, dependencies, and API logic into a standard format called a Bento.

Workflow

1. Save Model

import bentoml
bentoml.sklearn.save_model("my_clf", clf_obj)

2. Define Service (service.py)

import bentoml
from bentoml.io import NumpyNdarray

runner = bentoml.sklearn.get("my_clf:latest").to_runner()
svc = bentoml.Service("classifier", runners=[runner])

@svc.api(input=NumpyNdarray(), output=NumpyNdarray())
def predict(input_series):
    return runner.predict.run(input_series)

3. Build & Serve

bentoml build
bentoml serve service.py:svc

Why BentoML?

• Containerization: Automatically generates the Dockerfile for you.
• Adaptive Batching: Automatically groups API requests to maximize throughput.
• Yatai: A Kubernetes-native dashboard to manage deployments.
• Integration: Works with standard tools (MLflow) and deploys anywhere (AWS Lambda, SageMaker, Heroku, K8s).