Glossary · 6 · Quality, risk and governance
Evaluation (evals)
Also known as: Evals, LLM evaluation, Benchmark
Evaluation — “evals” for short — is the systematic measurement of how well an AI system performs a task, using a fixed set of test inputs with expected outputs or scoring criteria, scored by rules, by people or by another model, and repeated whenever the model, prompt or content changes.
- Expert
- Technical writers
- Technical project managers
- Developers
In one sentence
AI evaluation (evals) explained: measuring AI quality with test sets and scoring — the expert practice that makes AI projects controllable.
Example
Before launch, a team runs 200 real customer questions through its documentation assistant and checks whether each answer is correct, complete, cited and safe.
Explained in context
Context cards connect this term with others to answer one question. Also in British English and German.
- Why a copied interface is not a test boundaryWhat makes a test setup for an AI agent isolated — a copied interface or an enforced boundary?UKDeutsch
- Emotion and pain experiments on language models: what they measure and what they leave openWhen a study makes a language model “afraid”, “sad” or “in pain”, what has it shown — and what not?UKDeutsch
- Indirect prompt injection in agent chainsHow does indirect prompt injection travel through a chain of AI agents, and what changes when the chain gets control points?UKDeutsch
- Claims about AI consciousness: four questions that get mergedWhich different questions hide behind a sentence such as "the AI is not conscious"?UKDeutsch
- From behavior objective to evaluation criterionHow does a written behavior rule become a score in a model evaluation?UKDeutsch
Why it matters on your learning path
- Technical writers: Writers are ideal authors of eval sets: they know the real questions, the right answers and the edge cases.
- Technical project managers: No evals, no release. Define quality thresholds as acceptance criteria and track them over time.
- Developers: Automate evals in CI; combine exact checks, model-graded scoring and periodic human review.
Benchmarks vs. your own evals
Public benchmarks compare models on general tasks. Only evals on your content and questions tell you whether a system works for your users — see model selection.