Glossary · AI engineering and governance
Explainability (AI)
Also known as: Explainable AI, XAI
German: Erklärbarkeit
In AI, explainability is the property of an AI system to express the important factors influencing its results in a way that humans can understand. Explanations can be global (how the model works in general) or local (why a particular output was produced).
- Industrial AI
- AI
In one sentence
Explainability is an AI system's ability to express the main factors behind its results in a way that people can understand.
Example
Along with a reject decision, the inspection system highlights the image region with the suspected crack and shows the three features that contributed most.
How it applies
- Engineering: Explanation methods include feature importance, saliency maps and example-based explanations. They approximate the model's behavior and can themselves be misleading, so validate them as well.
- Operation: Operators and engineers need explanations to decide whether to trust a result, to find faults and to exercise meaningful Human oversight (AI Act).
- Documentation: Explain what the system's explanations mean and what they do not mean. Transparency information for users of high-risk AI systems, such as capabilities and limitations, belongs in the instructions for use.
Explainability vs. interpretability
Interpretability (AI) usually refers to how well a human can understand the model's internal mechanics, for example a small decision tree. Explainability refers to communicating the reasons for results, which is also possible for complex models using additional methods. Usage varies between authors.