LLM pricing and costs | ChipFoundryServices

Home› Knowledge Base› LLM pricing and costs

LLM pricing and costs are the economic factors that determine the total expense of running AI applications — including API costs per token, self-hosting infrastructure expenses, and optimization strategies, critical for building sustainable AI products and making build-vs-buy decisions.

What Are LLM Costs?

Definition: Total expense of using LLMs in production.
Components: API fees, infrastructure, optimization, engineering.
Unit: Typically cost per million tokens (input and output separately).
Variation: 100× difference between cheapest and most expensive options.

Why Pricing Matters

Product Economics: AI features must be profitable.
Build vs. Buy: Self-hosting vs. API decision.
Architecture Choices: Model routing, caching, batching decisions.
Scale Planning: Costs compound at scale.
Competitive Position: Lower costs enable lower prices or higher margins.

API Pricing Comparison (2024)

Provider/Model          | Input/1M tk | Output/1M tk | Notes
------------------------|-------------|--------------|---------------
GPT-4o                  | $2.50       | $10.00       | Most capable
GPT-4o-mini             | $0.15       | $0.60        | Cost-optimized
GPT-3.5-turbo           | $0.50       | $1.50        | Legacy
Claude 3.5 Sonnet       | $3.00       | $15.00       | Strong reasoning
Claude 3 Haiku          | $0.25       | $1.25        | Fast, cheap
Gemini 1.5 Pro          | $1.25       | $5.00        | Long context
Gemini 1.5 Flash        | $0.075      | $0.30        | Fastest
Llama 3.1 70B (hosted)  | $0.20-0.80  | $0.20-0.80   | Varies by host
Mistral Large           | $2.00       | $6.00        | European option

Self-Hosting Economics

Infrastructure Costs:

Hardware Option    | Monthly Cost | Models Served
-------------------|--------------|--------------------
RTX 4090 (24GB)    | ~$500 amort. | 7-13B models
A100 40GB          | $2-3K cloud  | Up to 30B
A100 80GB          | $3-4K cloud  | Up to 70B
H100 80GB          | $4-6K cloud  | 70B+ fast inference
8× H100 cluster    | $30-40K      | Any model, high throughput

Break-Even Analysis:

API cost example: $5/M tokens × 10M tokens/day = $50/day = $1,500/month
H100 cost: ~$5,000/month
Break-even: ~100M tokens/day for H100
Below this: API often cheaper
Above this: Self-host saves money

Cost Optimization Strategies

Caching:

Common queries → Cache responses
Hit rate of 20% → 20% cost reduction
Semantic caching: Similar queries hit cache
Implement: Redis, custom cache layer

Model Routing:

Simple queries → Cheap/small model (90% of traffic)
Complex queries → Expensive/large model (10% of traffic)
Potential savings: 60-80%

Prompt Optimization:

Before: 2,000 token system prompt
After: 500 token optimized prompt
Savings: 75% on input tokens

Techniques:
- Compression
- Remove redundancy
- Batch instructions

Output Control:

max_tokens: Set appropriate limits
Stop sequences: End early when possible
JSON mode: Structured output (often shorter)

Batching:

Real-time: Process individually (higher per-request cost)
Batch: Accumulate, process together (lower per-request cost)
When acceptable latency allows, batch for savings

Cost Tracking

What to Measure:

Tokens per request (input + output).
Requests per user/feature.
Cost per user action.
Cost per successful outcome.

Implementation:

class CostTracker:
    def __init__(self):
        self.costs = defaultdict(float)
    
    def record(self, user_id, feature, 
               input_tokens, output_tokens, model):
        cost = calculate_cost(
            input_tokens, output_tokens, model
        )
        self.costs[user_id] += cost
        self.costs[feature] += cost
        self.log(user_id, feature, cost)

Cost by Use Case

Use Case              | Typical Cost      | Optimization
----------------------|-------------------|-------------------
Chat (1 turn)         | $0.001-0.01       | Cache, small model
Code completion       | $0.0001-0.001     | Small model, prefix caching
Document summary      | $0.01-0.10        | Batch, smaller model
RAG (search + answer) | $0.005-0.05       | Cache embeddings
Agent (multi-step)    | $0.10-1.00        | Limit retries, cheaper tools

Cost Control Architecture

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">LLM Cost Optimization — Dollars per Million Tokens</text>
  <text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">model choice × quantization × caching × routing × batching = 10-100× cost reduction at equal quality</text>

  <!-- === TOP: Price landscape === -->
  <rect x="25" y="62" width="710" height="110" rx="6" fill="#080d14" stroke="#233043" stroke-width="1.2"/>
  <text x="380" y="80" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">API Pricing ($/1M output tokens, July 2026)</text>

  <!-- Price bars (conceptual, left to right = cheap to expensive) -->
  <rect x="45" y="94" width="40" height="55" rx="2" fill="#14261f" stroke="#34d399" stroke-width="0.7"/>
  <text x="65" y="118" fill="#6ee7b7" font-size="7" text-anchor="middle">Haiku</text>
  <text x="65" y="130" fill="#6ee7b7" font-size="8" text-anchor="middle">$1.25</text>

  <rect x="95" y="94" width="55" height="55" rx="2" fill="#14261f" stroke="#34d399" stroke-width="0.7"/>
  <text x="122" y="118" fill="#6ee7b7" font-size="7" text-anchor="middle">GPT-4o-mini</text>
  <text x="122" y="130" fill="#6ee7b7" font-size="8" text-anchor="middle">$0.60</text>

  <rect x="160" y="94" width="55" height="55" rx="2" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.7"/>
  <text x="187" y="118" fill="#93c5fd" font-size="7" text-anchor="middle">Sonnet 3.5</text>
  <text x="187" y="130" fill="#93c5fd" font-size="8" text-anchor="middle">$15</text>

  <rect x="225" y="94" width="50" height="55" rx="2" fill="#0f1a2a" stroke="#60a5fa" stroke-width="0.7"/>
  <text x="250" y="118" fill="#93c5fd" font-size="7" text-anchor="middle">GPT-4o</text>
  <text x="250" y="130" fill="#93c5fd" font-size="8" text-anchor="middle">$10</text>

  <rect x="285" y="94" width="50" height="55" rx="2" fill="#1a1520" stroke="#a78bfa" stroke-width="0.7"/>
  <text x="310" y="118" fill="#c4b5fd" font-size="7" text-anchor="middle">Opus</text>
  <text x="310" y="130" fill="#c4b5fd" font-size="8" text-anchor="middle">$75</text>

  <rect x="345" y="94" width="50" height="55" rx="2" fill="#1a0f0f" stroke="#f87171" stroke-width="0.7"/>
  <text x="370" y="118" fill="#f87171" font-size="7" text-anchor="middle">o1-pro</text>
  <text x="370" y="130" fill="#f87171" font-size="8" text-anchor="middle">$150</text>

  <!-- Self-hosted comparison -->
  <rect x="430" y="94" width="290" height="55" rx="4" fill="#0b1220" stroke="#233043" stroke-width="0.8"/>
  <text x="575" y="110" fill="#fbbf24" font-size="9" text-anchor="middle" font-weight="600">Self-hosted (vLLM on H100)</text>
  <text x="575" y="126" fill="#8b98a5" font-size="8.5" text-anchor="middle">Llama-3 70B: ~$0.40/MTok at full utilization</text>
  <text x="575" y="140" fill="#34d399" font-size="8.5" text-anchor="middle">Llama-3 8B quantized: ~$0.05/MTok</text>

  <text x="380" y="163" fill="#6b7684" font-size="8" text-anchor="middle">Output tokens cost 3-5× more than input tokens (decode is memory-bandwidth bound)</text>

  <!-- === MIDDLE LEFT: Cost levers === -->
  <rect x="25" y="180" width="350" height="135" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="200" y="198" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Cost Reduction Levers</text>

  <text x="45" y="218" fill="#34d399" font-size="8.5" font-weight="600">1. Model routing (5-10×)</text>
  <text x="205" y="218" fill="#8b98a5" font-size="8.5">cheap model for easy queries</text>
  <text x="45" y="236" fill="#60a5fa" font-size="8.5" font-weight="600">2. Prompt caching (2-5×)</text>
  <text x="200" y="236" fill="#8b98a5" font-size="8.5">reuse system prompt KV cache</text>
  <text x="45" y="254" fill="#fbbf24" font-size="8.5" font-weight="600">3. Batch API (2×)</text>
  <text x="155" y="254" fill="#8b98a5" font-size="8.5">async processing at 50% discount</text>
  <text x="45" y="272" fill="#c4b5fd" font-size="8.5" font-weight="600">4. Prompt compression (1.5-3×)</text>
  <text x="225" y="272" fill="#8b98a5" font-size="8.5">fewer input tokens (LLMLingua)</text>
  <text x="45" y="290" fill="#f87171" font-size="8.5" font-weight="600">5. Self-host + quantize (10-50×)</text>
  <text x="230" y="290" fill="#8b98a5" font-size="8.5">open model, INT4, full GPU util</text>
  <text x="45" y="308" fill="#8b98a5" font-size="8.5" font-weight="600">6. Distillation</text>
  <text x="125" y="308" fill="#8b98a5" font-size="8.5">train small model on large's outputs</text>

  <!-- === MIDDLE RIGHT: Cost math === -->
  <rect x="390" y="180" width="345" height="135" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="562" y="198" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Cost Math Example</text>

  <text x="410" y="220" fill="#8b98a5" font-size="8.5">Scenario: 1M requests/day, 500 in + 200 out tokens avg</text>
  <text x="410" y="240" fill="#f87171" font-size="8.5" font-weight="600">GPT-4o:</text>
  <text x="470" y="240" fill="#8b98a5" font-size="8.5">$2.5K input + $2K output = $4.5K/day</text>
  <text x="410" y="258" fill="#60a5fa" font-size="8.5" font-weight="600">GPT-4o-mini:</text>
  <text x="505" y="258" fill="#8b98a5" font-size="8.5">$75 input + $120 output = $195/day</text>
  <text x="410" y="276" fill="#34d399" font-size="8.5" font-weight="600">Self-hosted 8B:</text>
  <text x="510" y="276" fill="#8b98a5" font-size="8.5">~$35/day (1× H100 at $1.50/hr)</text>

  <text x="410" y="300" fill="#fbbf24" font-size="9">Saving: $4.5K → $195 (route) → $35 (self-host)</text>
  <text x="410" y="314" fill="#fbbf24" font-size="9">= 130× cost reduction, quality depends on task</text>

  <!-- === BOTTOM: Strategy === -->
  <rect x="25" y="325" width="710" height="78" rx="5" fill="#0b1220" stroke="#233043" stroke-width="1"/>
  <text x="380" y="343" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Optimization Strategy by Stage</text>

  <text x="130" y="367" fill="#34d399" font-size="9" text-anchor="middle" font-weight="600">Day 1: Prototype</text>
  <text x="130" y="381" fill="#8b98a5" font-size="8" text-anchor="middle">use best model, ignore cost</text>
  <text x="130" y="393" fill="#6b7684" font-size="7.5" text-anchor="middle">establish quality baseline</text>

  <text x="320" y="367" fill="#60a5fa" font-size="9" text-anchor="middle" font-weight="600">Week 2: Evaluate</text>
  <text x="320" y="381" fill="#8b98a5" font-size="8" text-anchor="middle">test smaller models on eval set</text>
  <text x="320" y="393" fill="#6b7684" font-size="7.5" text-anchor="middle">find cheapest at same quality</text>

  <text x="510" y="367" fill="#fbbf24" font-size="9" text-anchor="middle" font-weight="600">Month 1: Route</text>
  <text x="510" y="381" fill="#8b98a5" font-size="8" text-anchor="middle">classify queries, route to tiers</text>
  <text x="510" y="393" fill="#6b7684" font-size="7.5" text-anchor="middle">80% queries → mini model</text>

  <text x="670" y="367" fill="#c4b5fd" font-size="9" text-anchor="middle" font-weight="600">Month 3: Self-host</text>
  <text x="670" y="381" fill="#8b98a5" font-size="8" text-anchor="middle">fine-tune open model, deploy</text>
  <text x="670" y="393" fill="#6b7684" font-size="7.5" text-anchor="middle">maximize tokens/$/quality</text>

  <!-- Key insight -->
  <rect x="25" y="411" width="710" height="22" rx="3" fill="#0b1220" stroke="#233043" stroke-width="0.8"/>
  <text x="380" y="426" fill="#fbbf24" font-size="9" text-anchor="middle">The frontier model is the eval baseline, not the production model — always ask: can a cheaper model do this task?</text>

  <text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">LLM cost optimization is the new cloud cost optimization — the companies that solve it first win on unit economics.</text>
</svg>

LLM pricing and costs are the foundation of AI product economics — understanding and optimizing costs determines whether AI features are sustainable at scale, making cost engineering as important as prompt engineering for production AI systems.

costpricingtoken costbudgetapi pricingoptimizationself-hostingeconomics

Explore 500+ Semiconductor & AI Topics

From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.

🔍 Search Topics 💬 Ask CFSGPT 📚 Browse All

Related Topics

Explore 500+ Semiconductor & AI Topics