Skip to main content

Table 6 Descriptions and equations of metrics used during the analysis of the interpretations

From: Using test-time augmentation to investigate explainable AI: inconsistencies between method, model and human intuition

Metric

Analyses

Equation

Cosine distance

Distance between attributions

\(cos(\theta ) = 1-\frac{u \cdot v}{||u||_2||v||_2}\).

Entropy

Information contained in the attribution

\(H(\phi ) = - \sum _{i=0}^n \phi _i \times log_2(\phi _i)\)

Relative importance

Relative importance given to component

\(\sum \phi _i, \phi _i \in \text {component}\)

  1. Described are three metrics, what they measure and their equations. \(\phi _i\) is the attribution of token i. Entropy and cosine similarity were calculated only using the atom components