Glossary · AI engineering and governance
Interpretability (AI)
Also known as: Model interpretability
German: Interpretierbarkeit
In AI, interpretability is the degree to which a human can understand how a model arrives at its outputs from its structure and parameters, for example following the rules of a decision tree or the coefficients of a linear model.
- Industrial AI
- AI
In one sentence
Interpretability is the degree to which people can understand how a model reaches its outputs from its structure and parameters.
Example
A decision tree with six rules for predicting scrap can be read and checked line by line by process engineers, which makes it highly interpretable.
How it applies
- Engineering: Interpretable models, such as linear models, small trees or rule sets, can be checked against physical knowledge. In many industrial problems they perform nearly as well as complex models.
- Validation: Interpretability makes it easier to spot learned shortcuts, such as a model relying on a time stamp or a machine ID instead of the process signal.
- Documentation: When an interpretable model is used, document its rules or main parameters so that engineers can review them. For complex models, document the explanation methods used instead.
Interpretability vs. explainability
Interpretability is about understanding the model itself. Explainability (AI) is about communicating why the system produced a result, which can use additional methods for models that are not inherently interpretable. Many texts use the terms interchangeably.