Glossary · Automation software engineering and architecture
Mutation testing
Also known as: Mutation analysis
German: Mutationstest
In software testing, mutation testing evaluates the quality of a test suite by making small deliberate changes (mutants) to the code and checking whether the tests detect them; undetected mutants reveal weak tests.
- Software engineering
In one sentence
Mutation testing inserts small deliberate code changes and checks whether the test suite detects them, measuring test quality.
Example
A mutation tool changes a greater-than to greater-or-equal in the level monitoring function, and no unit test fails, so the team adds a boundary test.
How it applies
- Verification: Mutation testing answers a question Code coverage cannot: whether tests would actually notice a wrong result. Executed code is not necessarily checked code.
- Engineering: It is computationally expensive, so teams run it on critical modules or in nightly pipelines rather than on every commit.
- Documentation: When mutation scores are used as a quality metric, the test report should state the tool, the mutation operators and the treatment of equivalent mutants, so the figure can be interpreted.
Mutation testing vs. fault injection
Mutation testing changes the source code to assess the tests. Fault injection inserts faults into a running system, such as corrupted signals or failed components, to assess the system's fault reaction.