koboldcpp
**KoboldCpp** is a **single-file executable for running GGUF-quantized language models locally with a web UI optimized for creative writing and roleplay** — combining the llama.cpp inference engine with a purpose-built interface featuring context shifting (smart memory management for long conversations), World Info (automatic lore injection when keywords appear), and an immersive chat mode, all packaged into a zero-installation binary that runs on any platform.
**What Is KoboldCpp?**
- **Definition**: A self-contained local LLM inference application that bundles llama.cpp, a web server, and a creative-writing-focused UI into a single executable file — no Python, no Docker, no installation required. Download the binary, download a GGUF model, double-click, and start writing.
- **Creative Writing Focus**: While Ollama and LM Studio target general chat, KoboldCpp is specifically designed for interactive fiction, roleplay, and creative writing — with features like World Info, Author's Note, and Memory that are essential for maintaining narrative coherence in long stories.
- **Single Binary**: The entire application (inference engine, web server, UI) is compiled into one file — the simplest possible deployment for non-technical users who want to run AI locally.
- **KoboldAI Ecosystem**: KoboldCpp is the llama.cpp-based successor to KoboldAI (which used Transformers) — maintaining compatibility with the KoboldAI API and the large community of creative writing tools built around it.
**Key Features**
- **Context Shifting**: When the conversation exceeds the model's context window, KoboldCpp intelligently shifts context by removing older messages while preserving the system prompt, World Info, and recent conversation — maintaining coherence without abruptly truncating.
- **World Info**: Define lore entries with trigger keywords — when a keyword appears in the conversation, the associated lore text is automatically injected into the context. Essential for maintaining consistent world-building in interactive fiction.
- **Author's Note**: A persistent instruction injected near the end of the context — guides the model's writing style, tone, or plot direction without being visible in the conversation.
- **Memory**: A persistent text block always included at the top of the context — used for character descriptions, setting details, and story summaries that should always be available to the model.
- **Multiple Backends**: Supports GGUF (llama.cpp), with optional CUDA GPU acceleration, Vulkan GPU support, and CLBlast for OpenCL devices.
**KoboldCpp vs Alternatives**
| Feature | KoboldCpp | Ollama | SillyTavern + Backend | text-gen-webui |
|---------|----------|--------|----------------------|---------------|
| Installation | Single binary | CLI install | Node.js + backend | Python + pip |
| Creative writing | Excellent | Basic | Excellent (with ST) | Good |
| World Info | Built-in | No | Via SillyTavern | Extension |
| Context shifting | Built-in | Basic | Via SillyTavern | No |
| API compatibility | KoboldAI API | OpenAI API | Multiple | OpenAI API |
| Target user | Writers, RPers | Developers | Writers, RPers | Power users |
**KoboldCpp is the zero-installation creative writing AI tool** — packaging llama.cpp inference with narrative-focused features like World Info, context shifting, and immersive mode into a single executable that makes local AI storytelling accessible to anyone who can download two files.