Training
LoRA
Low-Rank Adaptation — a cheap way to fine-tune large AI models by training a small set of extra weights instead of the whole model.
01 ——
In plain English
LoRA (Low-Rank Adaptation) is a fine-tuning technique that lets you customise a large model without retraining all its billions of parameters. Instead, you train a small set of additional weights ("adapters") that modify the model's behaviour for your task.
Why LoRA matters:
- Cost — full fine-tuning a 70B model costs thousands; LoRA can cost dollars
- Storage — adapters are MBs, not GBs; you can have hundreds for one base model
- Speed — train in hours instead of days
- Composability — combine multiple LoRAs (one for tone, one for domain knowledge)
Where you'll see it:
- Image generation — Stable Diffusion communities share thousands of LoRAs for specific styles and subjects
- Open-weight LLMs — most fine-tunes of Llama, Mistral, etc. use LoRA
- Enterprise customisation — internal models tuned on company data
LoRA is the dominant approach to lightweight fine-tuning today.
02 ——
Related terms
Fine-tuning
Further training a pre-trained AI model on your own data to specialise it for a specific task or style.
Distillation
Training a smaller, cheaper AI model to mimic the outputs of a larger, more capable one — preserving most of the quality at a fraction of the cost.
Quantization
Shrinking an AI model by storing its weights in lower-precision numbers — making it smaller, faster, and cheaper with minimal quality loss.
Training Data
The dataset an AI model learns from — its quality, diversity, and biases directly shape what the model can do and how well it does it.
LLM
Large Language Model — the type of AI behind tools like ChatGPT and Claude, trained to understand and generate text.
Back to glossaryLast reviewed May 2026