Safety
Constitutional AI
An Anthropic-pioneered training method that teaches a model to critique and rewrite its own outputs against a written set of principles (a constitution).
01 ——
In plain English
Constitutional AI (CAI) is a training technique where you give the model a written set of rules — a "constitution" — and train it to follow them by self-critique. Instead of relying entirely on human feedback (RLHF) to flag bad outputs, the model also evaluates its own responses against the constitution and learns from those critiques.
How it works (simplified):
- Generate a response to a prompt
- Ask the model: "does this response violate any constitutional principle?"
- If yes, ask it to rewrite
- Use the original + rewrite as a training pair (RLAIF)
Why it matters:
- Reduces the number of human raters needed (faster, cheaper, more scalable)
- Makes the alignment criteria explicit and auditable
- Lets you update behaviour by changing the constitution, not the model
Origin: Introduced by Anthropic in 2022 and used to train Claude. The published constitution draws from sources like the UN Declaration of Human Rights, Apple's terms of service, and DeepMind's Sparrow rules.
02 ——
Related terms
RLHF
Reinforcement Learning from Human Feedback — the training technique that teaches AI models to be helpful, harmless, and honest.
RLAIF
Reinforcement Learning from AI Feedback — alignment training where another AI model, not a human, provides the preference signal used to fine-tune the target model.
Alignment
The challenge of making AI systems behave in ways that match human values and intentions — not just their literal instructions.
Guardrails
Rules and filters that constrain what an AI model can output — used to block harmful, off-topic, or non-compliant responses.
Red Teaming
Deliberately trying to make an AI model misbehave — find jailbreaks, exploits, and failure modes — before adversaries do.
Fine-tuning
Further training a pre-trained AI model on your own data to specialise it for a specific task or style.
Back to glossaryLast reviewed May 2026