Core concepts
Small Language Model
A compact language model — typically 1B to 15B parameters — designed to run cheaply, fast, or on-device while still being useful for focused tasks.
01 ——
In plain English
A small language model (SLM) is, well, a small one — usually billions instead of hundreds of billions of parameters. SLMs trade absolute capability for cost, latency, and the ability to run locally. As they've gotten dramatically better since 2024, they've become viable for many tasks that used to require a frontier model.
Popular SLM families (2025–26):
- Microsoft Phi — Phi-3, Phi-4 (3B–14B), trained heavily on synthetic data
- Google Gemma — 2B, 9B, 27B variants
- Meta Llama — Llama 3.2 1B/3B, Llama 4 Scout
- Alibaba Qwen — Qwen 2.5 0.5B through 7B
- Mistral Small / Ministral family
- Apple Foundation Model (on-device)
Where SLMs win:
- On-device — phones, laptops, IoT (no cloud round-trip, no API cost, full privacy)
- High-volume pipelines — millions of cheap classifications or extractions
- Latency-critical — voice agents, autocomplete
- Specialised tasks — fine-tuned SLMs often beat frontier models on the narrow task they were tuned for
The frontier-vs-small gap on routine tasks has shrunk faster than most predicted.
02 ——
Related terms
LLM
Large Language Model — the type of AI behind tools like ChatGPT and Claude, trained to understand and generate text.
Foundation Model
A large, general-purpose AI model trained on broad data that can be adapted (via prompting or fine-tuning) to many downstream tasks.
Quantization
Shrinking an AI model by storing its weights in lower-precision numbers — making it smaller, faster, and cheaper with minimal quality loss.
Distillation
Training a smaller, cheaper AI model to mimic the outputs of a larger, more capable one — preserving most of the quality at a fraction of the cost.
Knowledge Distillation
Training a small "student" model to imitate a large "teacher" model — capturing most of the teacher’s capability at a fraction of the size and cost.
Open-weight Model
An AI model whose trained weights are publicly released, so anyone can download, run, or fine-tune it themselves.
Back to glossaryLast reviewed May 2026