Metamorphic Testing is a software testing technique applied to ML models where test oracles are unavailable — instead of checking individual outputs, it verifies that known relationships (metamorphic relations) between inputs and outputs hold across transformations.
How Metamorphic Testing Works
- Metamorphic Relation: Define a known relationship: "if input $x$ is transformed to $T(x)$, then $f(T(x))$ should relate to $f(x)$ by relation $R$."
- Example: For a yield model, increasing temperature by 10°C while holding everything else constant should decrease yield by approximately $delta$ (domain knowledge).
- Test: Apply the transformation, run both inputs, and verify the relation holds.
- No Oracle Needed: You don't need to know the correct output — just that the relationship between outputs is correct.
Why It Matters
- Oracle Problem: For many ML tasks, the correct output is unknown — metamorphic testing sidesteps this.
- Domain Knowledge: Leverages engineering knowledge about how outputs should change with inputs.
- Process Models: Particularly valuable for semiconductor process models where physical relationships are known.
Metamorphic Testing is testing relationships, not outputs — verifying that known input-output relationships hold when the correct output itself is unknown.