Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models

Jiang, Dejun; Wu, Zhenxing; Hsieh, Chang-Yu; Chen, Guangyong; Liao, Ben; Wang, Zhe; Shen, Chao; Cao, Dongsheng; Wu, Jian; Hou, Tingjun

doi:10.1186/s13321-020-00479-8

Table 8 The performance comparison (Average AUC_ROC, MUV: Average AUC_PRC) of the 50 times independent runs on the five multi-task classification datasets for the eight models. (the top three model were italic for each dataset)

From: Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models

Dataset	No.	Tasks	Metric	Model	Training	Validation	Test
ClinTox	1475	2	AUC_ROC	SVM	0.922 ± 0.001	0.896 ± 0.048	0.888 ± 0.044
				XGBoost	0.985 ± 0.009	0.938 ± 0.035	0.911 ± 0.036
				RF	0.975 ± 0.003	0.918 ± 0.041	0.911 ± 0.042
				DNN	0.984 ± 0.014	0.929 ± 0.041	0.884 ± 0.051
				GCN	0.977 ± 0.020	0.945 ± 0.039	0.895 ± 0.046
				GAT	0.989 ± 0.010	0.941 ± 0.033	0.888 ± 0.042
				MPNN	0.895 ± 0.056	0.884 ± 0.069	0.847 ± 0.062
				Attentive FP	0.965 ± 0.018	0.943 ± 0.033	0.904 ± 0.043
SIDER	1366	27	AUC_ROC	SVM	0.953 ± 0.021	0.630 ± 0.025	0.630 ± 0.021
				XGBoost	0.954 ± 0.010	0.694 ± 0.023	0.642 ± 0.020
				RF	0.932 ± 0.001	0.655 ± 0.024	0.646 ± 0.022
				DNN	0.814 ± 0.064	0.657 ± 0.029	0.631 ± 0.028
				GCN	0.902 ± 0.047	0.656 ± 0.021	0.634 ± 0.026
				GAT	0.865 ± 0.068	0.663 ± 0.024	0.627 ± 0.024
				MPNN	0.741 ± 0.010	0.637 ± 0.030	0.598 ± 0.031
				Attentive FP	0.834 ± 0.103	0.657 ± 0.024	0.623 ± 0.026
Tox21	7811	12	AUC_ROC	SVM	0.972 ± 0.001	0.821 ± 0.013	0.817 ± 0.009
				XGBoost	0.989 ± 0.005	0.857 ± 0.009	0.836 ± 0.010
				RF	0.981 ± 0.001	0.840 ± 0.010	0.838 ± 0.011
				DNN	0.920 ± 0.022	0.849 ± 0.012	0.840 ± 0.014
				GCN	0.961 ± 0.019	0.846 ± 0.013	0.836 ± 0.016
				GAT	0.946 ± 0.025	0.842 ± 0.013	0.835 ± 0.014
				MPNN	0.896 ± 0.023	0.826 ± 0.014	0.809 ± 0.017
				Attentive FP	0.939 ± 0.021	0.859 ± 0.012	0.852 ± 0.012
ToxCast	8539	182	AUC_ROC	SVM	0.982 ± 0.007	0.723 ± 0.005	0.722 ± 0.006
				XGBoost	0.976 ± 0.002	0.800 ± 0.004	0.774 ± 0.004
				RF	0.949 ± 0.000	0.783 ± 0.005	0.782 ± 0.005
				DNN	0.900 ± 0.021	0.797 ± 0.017	0.786 ± 0.019
				GCN	0.891 ± 0.020	0.784 ± 0.019	0.770 ± 0.016
				GAT	0.881 ± 0.021	0.782 ± 0.018	0.768 ± 0.018
				MPNN	0.802 ± 0.033	0.746 ± 0.022	0.731 ± 0.021
				Attentive FP	0.921 ± 0.037	0.804 ± 0.020	0.794 ± 0.017
MUV	93087	17	AUC_PRC	SVM	0.834 ± 0.046	0.107 ± 0.036	0.112 ± 0.045
				XGBoost	0.646 ± 0.064	0.095 ± 0.039	0.068 ± 0.028
				RF	0.704 ± 0.019	0.053 ± 0.024	0.061 ± 0.032
				DNN	0.027 ± 0.028	0.030 ± 0.031	0.021 ± 0.030
				GCN	0.182 ± 0.012	0.067 ± 0.030	0.061 ± 0.034
				GAT	0.151 ± 0.078	0.062 ± 0.028	0.057 ± 0.030
				MPNN	0.011 ± 0.005	0.024 ± 0.022	0.016 ± 0.010
				Attentive FP	0.066 ± 0.052	0.040 ± 0.034	0.038 ± 0.024

Back to article page

ISSN: 1758-2946

Contact us

Submission enquiries: journalsubmissions@springernature.com

Journal of Cheminformatics

Contact us