gorilla
**Gorilla** is a **fine-tuned large language model specifically trained to generate accurate API calls, solving the critical problem of LLM hallucination when generating code for complex APIs** — trained on a comprehensive dataset of API documentation from thousands of machine learning APIs (Hugging Face, PyTorch Hub, TensorFlow Hub), Gorilla generates syntactically correct function calls with proper parameters, types, and constraints that can be executed directly without the hallucinated arguments and invented parameters that plague general-purpose models.
**What Is Gorilla?**
- **Definition**: A Llama-based LLM fine-tuned by UC Berkeley researchers on API documentation to accurately generate executable API calls — addressing the specific failure mode where general LLMs hallucinate plausible-sounding but non-existent API parameters, wrong argument types, or deprecated function signatures.
- **The Hallucination Problem**: When asked to "load a BERT model for sentiment analysis using Hugging Face," general LLMs (GPT-4, Llama) often generate calls with wrong model names, deprecated parameters, or invented arguments that look correct but fail at runtime. Gorilla eliminates this by training on actual API documentation.
- **API Coverage**: Trained on documentation from Hugging Face Model Hub (1,645 models), PyTorch Hub (117 models), TensorFlow Hub (802 models), and extensible to any documented API — covering model loading, inference, and configuration calls.
- **Retrieval-Augmented Generation**: Gorilla optionally retrieves current API documentation at inference time — enabling it to stay updated as APIs change versions without retraining.
**How Gorilla Works**
| Step | Process | Benefit |
|------|---------|---------|
| 1. User prompt | "Load a text-to-image model that runs on single GPU" | Natural language intent |
| 2. API retrieval | Fetch relevant documentation | Current parameter info |
| 3. Constraint matching | Filter by hardware/license requirements | Practical constraints |
| 4. Code generation | Generate exact API call with correct params | Executable output |
| 5. Validation | Verify against API schema | No hallucinated args |
**Performance**
| Metric | Gorilla | GPT-4 | Claude | LLaMA-7B |
|--------|---------|-------|--------|----------|
| API Call Accuracy | **90.1%** | 72.8% | 68.5% | 32.1% |
| Hallucination Rate | **4.2%** | 24.7% | 28.1% | 61.3% |
| Executable Output | **88.3%** | 65.1% | 59.2% | 18.4% |
| Correct Parameters | **92.7%** | 71.3% | 67.8% | 28.9% |
**Key Innovation**: Gorilla achieves nearly **6× lower hallucination rate** than GPT-4 on API generation tasks — the difference between code that runs and code that crashes with "argument not found" errors.
**Significance**
- **Tool Use Foundation**: Gorilla demonstrated that LLMs can be trained to reliably interact with external tools and APIs — a prerequisite for autonomous AI agents that need to execute real-world actions.
- **AST Evaluation**: Introduced Abstract Syntax Tree (AST) evaluation for generated API calls — checking structural correctness rather than just string matching, establishing a rigorous evaluation methodology.
- **Continual Updates**: The retrieval-augmented approach allows Gorilla to adapt to API changes without retraining — critical for production systems where APIs are versioned and updated frequently.
**Gorilla is the pioneering API-specialized LLM that proved language models can be trained to generate reliable, executable code for complex APIs** — reducing hallucination rates by 6× compared to general-purpose models and establishing the foundation for autonomous AI agents that interact with real-world software systems.