bloom

BLOOM (BigScience Large Open-science Open-access Multilingual Language Model) is a 176 billion parameter open-source multilingual language model created by the BigScience research workshop — a year-long collaboration of over 1,000 researchers from 60+ countries and 250+ institutions, representing the largest open scientific collaboration for LLM development. Released in 2022, BLOOM is notable for its commitment to multilingual capability, open science, and ethical AI development. BLOOM's multilingual design sets it apart from other large models: it was trained on ROOTS (Responsible Open-science Open-collaboration Text Sources), a 1.6 TB curated dataset covering 46 natural languages (including many underrepresented languages — Swahili, Yoruba, Igbo, Fon, Wolof, and other African languages alongside European, Asian, and other language families) and 13 programming languages. This deliberate linguistic diversity aims to make LLM capabilities accessible beyond the English-dominant training paradigm. Architecture: BLOOM uses a decoder-only transformer with ALiBi positional embeddings (enabling context length generalization) and embedding layer normalization. Training was conducted on the Jean Zay supercomputer in France using 384 NVIDIA A100 80GB GPUs over approximately 3.5 months. BLOOM was among the first 100B+ parameter models released with fully open weights and detailed documentation of training data, methodology, carbon emissions, and governance processes. The BigScience project also produced the BLOOMZ variant (fine-tuned on crosslingual task data for improved zero-shot multilingual performance). BLOOM's governance structure introduced the Responsible AI License (RAIL), which allows broad use but prohibits specific harmful applications — a middle ground between fully open licenses and proprietary restrictions. While BLOOM has been surpassed in performance by later models, its contributions to open, collaborative, and ethically intentional AI development remain influential in how large models are developed and released.