helicone
**Helicone** is an **open-source LLM observability platform that adds comprehensive logging, caching, rate limiting, and cost tracking to any LLM application through a one-line proxy configuration change** — providing the monitoring infrastructure that production AI applications need without requiring SDK changes, custom middleware, or complex instrumentation.
**What Is Helicone?**
- **Definition**: An open-source observability proxy (cloud-hosted at helicone.ai or self-hosted) that intercepts OpenAI, Anthropic, Azure, and other LLM API calls — recording every request and response in real-time with full metadata, then forwarding to the actual provider.
- **One-Line Integration**: Change `base_url` in your existing SDK from `https://api.openai.com/v1` to `https://oai.helicone.ai/v1` and add your Helicone API key as a header — no other code changes required, all existing calls are instantly instrumented.
- **Open Source**: The Helicone codebase is public (Apache 2.0 license) — self-host on your own infrastructure for complete data sovereignty, or use the managed cloud version for zero-ops setup.
- **Real-Time Dashboard**: Every LLM call appears in the Helicone dashboard within seconds — live monitoring of request volume, latency, error rates, and cost without batch processing delays.
- **Custom Properties**: Attach metadata to any request via headers (`Helicone-Property-User-Id`, `Helicone-Property-Session`) — slice any metric by user, feature, experiment, or any custom dimension.
**Why Helicone Matters**
- **Instant Visibility**: Go from zero observability to full request logging in under 60 seconds — no instrumentation code, no logging pipeline, no data warehouse setup required.
- **Cost Control**: Per-request cost tracking with USD amounts — "Which users are costing the most?" "Which prompts are the most expensive?" answered immediately from the dashboard.
- **Caching for Cost Reduction**: Built-in exact-match and semantic caching can reduce API costs by 20-50% for applications with repeated queries — saved responses return in milliseconds at zero API cost.
- **Rate Limiting**: Protect your API keys from abuse with per-user rate limits — prevent a single user from consuming your entire monthly API budget with a runaway loop.
- **Debugging Production Issues**: When users report wrong answers, replay the exact request (with the same input, model, and parameters) from the Helicone dashboard — reproduce production bugs without access to application logs.
**Core Helicone Features**
**Zero-Code Integration**:
```python
from openai import OpenAI
client = OpenAI(
api_key="sk-...",
base_url="https://oai.helicone.ai/v1",
default_headers={"Helicone-Auth": "Bearer pk-helicone-..."}
)
# All subsequent API calls are automatically logged
```
**For Anthropic**:
```python
import anthropic
client = anthropic.Anthropic(
api_key="sk-ant-...",
base_url="https://anthropic.helicone.ai",
default_headers={"Helicone-Auth": "Bearer pk-helicone-..."}
)
```
**Custom Properties for Segmentation**:
```python
client = OpenAI(
base_url="https://oai.helicone.ai/v1",
default_headers={
"Helicone-Auth": "Bearer pk-helicone-...",
"Helicone-Property-User-Id": "user_123",
"Helicone-Property-Feature": "document-summarizer",
"Helicone-Property-Environment": "production"
}
)
```
**Caching**:
```python
default_headers={
"Helicone-Auth": "Bearer pk-helicone-...",
"Helicone-Cache-Enabled": "true", # Enable caching
"Helicone-Cache-Bucket-Max-Size": "5", # Cache up to 5 responses per prompt
}
```
**Rate Limiting**:
```python
default_headers={
"Helicone-RateLimit-Policy": "10;w=60;s=user", # 10 requests per 60s per user
"Helicone-User-Id": "user_123"
}
```
**Observability Dashboard Features**
- **Request Explorer**: Search and filter all requests by model, user, date, cost, latency, or custom property — find the exact request that caused an issue.
- **Aggregate Metrics**: Daily active users, average latency by model, total tokens consumed, total cost — track key health metrics over time.
- **Prompt Templates**: Group requests by prompt template for comparative analysis — see which prompt version has better latency or lower error rate.
- **Session Tracking**: Group related requests into sessions — trace a full multi-turn conversation as a single unit.
- **Evaluation Scores**: Attach quality scores to requests via the API — track model output quality alongside cost and latency.
**Helicone vs Alternatives**
| Feature | Helicone | Langfuse | Portkey | DataDog LLM |
|---------|---------|---------|---------|------------|
| Setup complexity | Minimal | Low | Low | High |
| Open source | Yes | Yes | Partial | No |
| Caching | Yes | No | Yes | No |
| Rate limiting | Yes | No | Yes | No |
| Provider support | OpenAI, Anthropic, Azure | OpenAI, Anthropic | 200+ | OpenAI |
| Self-hostable | Yes | Yes | Enterprise | No |
Helicone is **the fastest path from an un-monitored LLM application to full production observability** — its proxy architecture means any team can add comprehensive logging, cost tracking, and caching to their AI application in minutes, without modifying application code or building custom instrumentation infrastructure.