Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models

Table 7 The performance comparison (Average AUC_ROC) of the 50 times independent runs on the three classification datasets for the eight models. (the top three model were italic for each dataset)

Dataset	No.	Tasks	Metric	Model	Training	Validation	Test
HIV	40748	1	AUC_ROC	SVM	1.000 ± 0.000	0.825 ± 0.023	0.822 ± 0.020
				XGBoost	0.990 ± 0.012	0.831 ± 0.022	0.816 ± 0.020
				RF	0.963 ± 0.002	0.819 ± 0.021	0.820 ± 0.016
				DNN	0.935 ± 0.040	0.825 ± 0.020	0.797 ± 0.018
				GCN	0.984 ± 0.024	0.852 ± 0.023	0.834 ± 0.025
				GAT	0.957 ± 0.036	0.841 ± 0.019	0.826 ± 0.030
				MPNN	0.934 ± 0.040	0.828 ± 0.022	0.811 ± 0.031
				Attentive FP	0.928 ± 0.052	0.839 ± 0.022	0.822 ± 0.026
BACE	1513	1	AUC_ROC	SVM	0.979 ± 0.002	0.891 ± 0.026	0.893 ± 0.020
				XGBoost	0.994 ± 0.010	0.903 ± 0.029	0.889 ± 0.021
				RF	0.988 ± 0.001	0.896 ± 0.031	0.890 ± 0.022
				DNN	0.976 ± 0.015	0.916 ± 0.024	0.890 ± 0.024
				GCN	0.990 ± 0.018	0.921 ± 0.025	0.898 ± 0.019
				GAT	0.981 ± 0.021	0.916 ± 0.024	0.886 ± 0.023
				MPNN	0.926 ± 0.028	0.876 ± 0.030	0.838 ± 0.027
				Attentive FP	0.970 ± 0.029	0.906 ± 0.033	0.876 ± 0.023
BBBP	2035	1	AUC_ROC	SVM	0.988 ± 0.002	0.919 ± 0.029	0.919 ± 0.028
				XGBoost	0.995 ± 0.005	0.938 ± 0.022	0.926 ± 0.026
				RF	0.990 ± 0.001	0.929 ± 0.026	0.927 ± 0.025
				DNN	0.990 ± 0.010	0.938 ± 0.022	0.922 ± 0.029
				GCN	0.981 ± 0.018	0.931 ± 0.024	0.903 ± 0.027
				GAT	0.987 ± 0.016	0.927 ± 0.022	0.898 ± 0.033
				MPNN	0.961 ± 0.024	0.916 ± 0.030	0.879 ± 0.037
				Attentive FP	0.972 ± 0.021	0.922 ± 0.027	0.887 ± 0.032

ISSN: 1758-2946