Glossary · AI engineering and governance
Explainability (AI)
Also known as: Explainable AI, XAI
German: Erklärbarkeit
In AI, explainability is the property of an AI system to express the important factors influencing its results in a way that humans can understand. Explanations can be global (how the model works in general) or local (why a particular output was produced).
- Industrial AI
- AI
In one sentence
Explainability is an AI system's ability to express the main factors behind its results in a way that people can understand.
Example
Along with a reject decision, the inspection system highlights the image region with the suspected crack and shows the three features that contributed most.
Explained in context
Context cards connect this term with others to answer one question. Also in British English and German.
How it applies
- Engineering: Explanation methods include feature importance, saliency maps and example-based explanations. They approximate the model's behavior and can themselves be misleading, so validate them as well.
- Operation: Operators and engineers need explanations to decide whether to trust a result, to find faults and to exercise meaningful Human oversight (AI Act).
- Documentation: Explain what the system's explanations mean and what they do not mean. Transparency information for users of high-risk AI systems, such as capabilities and limitations, belongs in the instructions for use.
Explainability vs. interpretability
Interpretability (AI) usually refers to how well a human can understand the model's internal mechanics, for example a small decision tree. Explainability refers to communicating the reasons for results, which is also possible for complex models using additional methods. Usage varies between authors.