polycoder

**PolyCoder** is a **completely open-source code generation model developed at Carnegie Mellon University, trained on 249GB of code across 12 programming languages from GitHub** — pioneering fully transparent AI research where code, weights, and training data are all public, challenging OpenAI's proprietary Codex dominance and proving that reproducible research could compete with closed commercial systems. **Architecture & Training Philosophy** | Component | Specification | |-----------|--------------| | **Parameters** | 2.7B (decoder-only transformer) | | **Training Data** | 249GB of GitHub code (C, C++, Java, Python, JS, Go, Ruby, Rust, etc.) | | **License** | Fully open: weights + training code + data methodology public | | **Languages** | 12 primary languages with balanced representation | PolyCoder's radical openness was revolutionary at release (early 2022) when code generation was dominated by restricted APIs. **Key Achievement**: Achieved surprising superiority in **C language generation** compared to Codex despite being much smaller—proving that careful data composition (proportional language representation) matters more than raw scale. **Significance**: Established that **fully open, reproducible code generation** was both feasible and valuable, enabling independent researchers to study optimization techniques without API access. While newer models (StarCoder, Code Llama) surpassed PolyCoder in capability, its role as the first truly open code model made it a foundational milestone in democratizing AI research.