replit code
**Replit Code** is a **3 billion parameter code generation model developed by Replit, optimized specifically for low-latency IDE autocompletion with sub-100ms suggestion times** — trained on a combination of permissively licensed code from The Stack and Replit's proprietary corpus of billions of lines written on the Replit platform, powering the "Ghostwriter" AI assistant that provides real-time code completion directly within the Replit online development environment.
---
**Architecture & Training**
| Component | Detail |
|-----------|--------|
| **Parameters** | 2.7B (replit-code-v1-3b) |
| **Architecture** | Decoder-only transformer optimized for streaming inference |
| **Training Data** | The Stack (permissive licenses) + Replit proprietary code corpus |
| **Context Window** | 16,384 tokens (large for its size class) |
| **Tokenizer** | Custom code-aware tokenizer with whitespace/indentation tokens |
| **Languages** | 20 programming languages with emphasis on Python, JavaScript, TypeScript |
The custom tokenizer is critical — standard NLP tokenizers waste tokens on whitespace and indentation. Replit's tokenizer treats common indentation patterns as single tokens, effectively doubling the useful context length for code.
---
**Latency-First Design**
Unlike research models optimized for benchmark scores, Replit Code was engineered for **production latency constraints**:
**The IDE Problem**: Code suggestions must appear in under 100ms — any slower and they feel laggy and disruptive to the developer's flow. This constraint eliminates most large models (7B+) from consideration for on-device deployment.
**Ghostwriter Integration**: The model runs on Replit's cloud infrastructure with optimized inference (quantization, speculative decoding, KV-cache optimization) to serve suggestions to millions of concurrent developers with consistent sub-100ms latency.
**16K Context Advantage**: The large context window allows Ghostwriter to read the entire current file plus imported modules, producing suggestions that are contextually aware of the broader codebase rather than just the immediate cursor position.
---
**Significance**
Replit Code represents the **"production-first"** approach to code models — prioritizing real-world deployment constraints (latency, cost, concurrent users) over academic benchmarks. While it scores lower than Code Llama 34B on HumanEval, it delivers a superior user experience in practice because suggestions arrive instantly.
This philosophy — that a fast 3B model in production beats a slow 34B model in theory — influenced the entire industry's shift toward **Small Language Models (SLMs)** for code completion, including Microsoft's Phi series and Google's Gemma for on-device deployment.