Using test-time augmentation to investigate explainable AI: inconsistencies between method, model and human intuition

Hartog, Peter B. R.; Krüger, Fabian; Genheden, Samuel; Tetko, Igor V.

doi:10.1186/s13321-024-00824-1

Table 10 AUROC, accuracy, F1, MCC precision and recall scores with bootstrap variability of MLP models transfer learned on Ames data

From: Using test-time augmentation to investigate explainable AI: inconsistencies between method, model and human intuition

Architecture	Training	AUROC\(\uparrow\)	Accuracy\(\uparrow\)	F1\(\uparrow\)	MCC\(\uparrow\)	Precision\(\uparrow\)	Recall\(\uparrow\)
No training	Untrained	0.640 ± 0.004	0.518 ± 0.003	0.172 ± 0.008	0.066 ± 0.011	0.100 ± 0.005	0.610 ± 0.019
	Native	0.652 ± 0.005	0.611 ± 0.006	0.601 ± 0.006	0.221 ± 0.012	0.587 ± 0.006	0.616 ± 0.006
Variations	Random split	0.853 ± 0.004	0.776 ± 0.006	0.776 ± 0.006	0.551 ± 0.012	0.776 ± 0.006	0.775 ± 0.006
	Training set	0.872 ± 0.002	0.789 ± 0.003	0.789 ± 0.003	0.577 ± 0.007	0.789 ± 0.003	0.788 ± 0.003
	CNN	0.721 ± 0.006	0.663 ± 0.008	0.668 ± 0.008	0.326 ± 0.016	0.678 ± 0.008	0.658 ± 0.008
	Enumerated	0.812 ± 0.003	0.740 ± 0.005	0.740 ± 0.005	0.480 ± 0.011	0.739 ± 0.005	0.741 ± 0.005
Encoder only	C2C	0.735 ± 0.004	0.666 ± 0.006	0.665 ± 0.006	0.332 ± 0.011	0.665 ± 0.006	0.666 ± 0.006
	R2C	0.735 ± 0.004	0.672 ± 0.006	0.672 ± 0.005	0.344 ± 0.011	0.672 ± 0.006	0.672 ± 0.006
	E2C	0.733 ± 0.004	0.657 ± 0.005	0.657 ± 0.005	0.315 ± 0.011	0.657 ± 0.005	0.658 ± 0.005
	MC2C	0.764 ± 0.004	0.695 ± 0.006	0.696 ± 0.006	0.389 ± 0.012	0.698 ± 0.006	0.693 ± 0.006
	MR2C	0.694 ± 0.003	0.648 ± 0.004	0.647 ± 0.004	0.296 ± 0.009	0.644 ± 0.004	0.649 ± 0.004
	ME2C	0.808 ± 0.003	0.734 ± 0.006	0.734 ± 0.006	0.468 ± 0.012	0.733 ± 0.006	0.735 ± 0.006
Encoder-decoder	C2C	0.715 ± 0.002	0.660 ± 0.005	0.661 ± 0.005	0.321 ± 0.009	0.663 ± 0.005	0.660 ± 0.005
	R2C	0.699 ± 0.005	0.640 ± 0.007	0.640 ± 0.007	0.281 ± 0.013	0.640 ± 0.007	0.640 ± 0.007
	E2C	0.748 ± 0.003	0.677 ± 0.006	0.678 ± 0.005	0.355 ± 0.011	0.680 ± 0.006	0.677 ± 0.006
	MC2C	0.748 ± 0.004	0.683 ± 0.006	0.682 ± 0.006	0.366 ± 0.012	0.681 ± 0.006	0.684 ± 0.006
	MR2C	0.687 ± 0.006	0.633 ± 0.007	0.633 ± 0.007	0.267 ± 0.013	0.633 ± 0.007	0.634 ± 0.007
	ME2C	0.782 ± 0.005	0.710 ± 0.007	0.710 ± 0.007	0.420 ± 0.014	0.711 ± 0.007	0.709 ± 0.007

Values are based on the scaffold split. ± values have been determined using 1000 fold test-time bootstrapping

Back to article page

ISSN: 1758-2946

Contact us

Submission enquiries: journalsubmissions@springernature.com

Journal of Cheminformatics

Contact us