OpenAI Embeddings

Overview OpenAI provides API-based embedding models that convert text into vector representations. They are the industry standard for "getting started" with RAG (Retrieval Augmented Generation) due to their ease of use, decent performance, and high context window.

Models

1. text-embedding-3-small (New Standard)

• Cost: Extremely cheap ($0.00002 / 1k tokens).
• Dimensions: 1536 (default), but can be shortened.
• Performance: Better than Ada-002.

2. text-embedding-3-large

• Performance: SOTA performance for English retrieval.
• Dimensions: 3072.
• Use Case: When accuracy matters more than cost/storage.

3. text-embedding-ada-002 (Legacy)

• The workhorse model used in most tutorials from 2023. Still supported but 3-small is better and cheaper.

Dimensions & Matryoshka Learning The new v3 models support shortening embeddings (e.g., from 1536 to 256) without losing much accuracy. This saves massive amounts of storage in your vector database.

Usage

from openai import OpenAI
client = OpenAI()

response = client.embeddings.create(
    input="The food was delicious",
    model="text-embedding-3-small"
)

vector = response.data[0].embedding
**[0.0023, -0.012, ...]**

Comparison

• Pros: Easy API, high reliability, large context (8k tokens).
• Cons: Cost (at scale), data privacy (cloud), "black box" training.