Skip to main content

Table 2 AUC ROC for all classification techniques and AD measures

From: Efficiency of different measures for defining the applicability domain of classification models

  MUSK2 QSAR BBB PGP FXa Liver hERG Cancer Ames CYP1A2 Mean rank #Signif.a
RF
\(\bar{p}_{RFC}\) 0.99 b 0.93 0.86 0.85 0.98 0.59 0.86 0.64 0.87 0.90 1.95 10
\(PROBSTD_{RFR}\) 0.99 0.93 0.85 0.85 0.98 0.59 0.86 0.64 0.86 0.90 2.25 10
\(\bar{p}_{RFR}\) 0.99 0.93 0.85 0.85 0.97 0.59 0.86 0.64 0.86 0.90 2.60 10
\(STD_{RFR}\) 0.99 0.93 0.84 0.84 0.98 0.58 0.85 0.63 0.85 0.90 3.45 8
\(\cos_{\alpha }\) 0.95 0.87 0.82 0.82 0.97 0.56 0.78 0.61 0.81 0.84 6.10 5
\(\gamma_{Euc}\) 0.95 0.86 0.82 0.81 0.97 0.55 0.79 0.61 0.79 0.85 6.50 4
\(\kappa_{Euc}\) 0.94 0.86 0.81 0.80 0.97 0.54 0.80 0.61 0.79 0.85 7.10 4
\(\delta_{Euc}\) 0.94 0.86 0.84 0.79 0.96 0.58 0.78 0.59 0.80 0.82 7.45 1
\(\delta_{\text{Tan}}\) 0.92 0.85 0.79 0.80 0.95 0.55 0.77 0.60 0.78 0.83 9.10 0
\(\gamma_{\text{Tan}}\) 0.91 0.85 0.78 0.80 0.94 0.56 0.78 0.58 0.78 0.81 9.70 0
\(\kappa_{\text{Tan}}\) 0.92 0.85 0.78 0.79 0.94 0.57 0.76 0.59 0.78 0.81 9.80 0
Range 0.08 0.08 0.08 0.06 0.04 0.05 0.10 0.06 0.09 0.09   
NN
\(\bar{p}_{NNR}\) 1.00 0.92 0.83 0.84 0.98 0.57 0.81 0.62 0.84 0.89 2.00 10
\(\bar{p}_{NNC}\) 1.00 0.92 0.83 0.84 0.98 0.56 0.82 0.61 0.84 0.89 2.25 9
\(PROBSTD_{NNR}\) 1.00 0.92 0.82 0.83 0.98 0.57 0.81 0.62 0.84 0.89 2.30 10
\(STD_{NNR}\) 1.00 0.92 0.79 0.82 0.98 0.56 0.80 0.61 0.83 0.88 3.50 9
\(\gamma_{Euc}\) 0.99 0.86 0.76 0.82 0.96 0.52 0.77 0.59 0.77 0.85 6.65 4
\(\kappa_{Euc}\) 0.99 0.86 0.76 0.81 0.96 0.52 0.78 0.59 0.77 0.85 6.75 4
\(\cos_{\alpha }\) 0.99 0.88 0.76 0.76 0.96 0.53 0.73 0.59 0.79 0.83 7.15 4
\(\delta_{\text{Tan}}\) 0.97 0.85 0.75 0.81 0.95 0.53 0.76 0.58 0.75 0.82 8.70 2
\(\delta_{Euc}\) 0.98 0.86 0.78 0.73 0.95 0.54 0.74 0.58 0.77 0.81 8.05 0
\(\kappa_{\text{Tan}}\) 0.96 0.86 0.76 0.76 0.93 0.55 0.74 0.57 0.75 0.80 9.10 0
\(\gamma_{\text{Tan}}\) 0.96 0.85 0.75 0.77 0.93 0.55 0.74 0.57 0.75 0.80 9.55 0
Range 0.04 0.07 0.08 0.11 0.05 0.05 0.09 0.05 0.09 0.09   
SVM
\(\hat{p}_{SVR}\) 1.00 0.90 0.86 0.84 0.97 0.57 0.83 0.61 0.84 0.87 1.50 9
\(\hat{p}_{SVC}\) 1.00 0.91 0.87 0.82 0.97 0.56 0.83 0.60 0.84 0.88 1.60 9
\(\cos_{\alpha }\) 0.99 0.87 0.80 0.79 0.95 0.56 0.77 0.59 0.79 0.82 4.30 6
\(\gamma_{Euc}\) 0.99 0.86 0.82 0.75 0.95 0.54 0.79 0.59 0.78 0.83 4.35 5
\(\kappa_{Euc}\) 0.99 0.86 0.82 0.74 0.95 0.53 0.79 0.59 0.78 0.83 4.75 5
\(\delta_{Euc}\) 0.98 0.85 0.83 0.74 0.93 0.56 0.75 0.58 0.77 0.80 5.80 0
\(\delta_{\text{Tan}}\) 0.98 0.85 0.81 0.74 0.92 0.53 0.76 0.58 0.76 0.80 6.80 0
\(\kappa_{\text{Tan}}\) 0.97 0.84 0.82 0.72 0.91 0.55 0.75 0.57 0.75 0.78 7.70 0
\(\gamma_{\text{Tan}}\) 0.97 0.84 0.81 0.72 0.90 0.54 0.75 0.57 0.75 0.78 8.20 0
Range 0.03 0.07 0.07 0.12 0.07 0.04 0.08 0.04 0.09 0.10   
MB
\(\bar{f}_{MB}\) 0.93 0.90 0.80 0.84 0.97 0.59 0.83 0.58 0.76 0.85 1.20 10
\(\hat{v}_{MB}\) 0.93 0.89 0.79 0.83 0.97 0.58 0.83 0.58 0.75 0.82 1.80 9
\(\cos_{\alpha }\) 0.89 0.86 0.75 0.80 0.95 0.57 0.74 0.56 0.74 0.80 4.20 6
\(\kappa_{Euc}\) 0.90 0.84 0.73 0.73 0.95 0.55 0.77 0.56 0.72 0.80 4.85 4
\(\gamma_{Euc}\) 0.89 0.84 0.73 0.74 0.95 0.55 0.76 0.56 0.72 0.80 5.10 4
\(\delta_{Euc}\) 0.87 0.83 0.75 0.74 0.93 0.57 0.77 0.55 0.70 0.75 5.30 1
\(\delta_{\text{Tan}}\) 0.86 0.82 0.69 0.73 0.92 0.56 0.76 0.55 0.69 0.75 6.90 0
\(\gamma_{\text{Tan}}\) 0.81 0.79 0.69 0.72 0.89 0.56 0.77 0.54 0.66 0.72 7.80 0
\(\kappa_{\text{Tan}}\) 0.81 0.79 0.69 0.72 0.89 0.57 0.76 0.54 0.66 0.72 7.85 0
Range 0.12 0.11 0.11 0.12 0.08 0.04 0.09 0.04 0.10 0.13   
k-NN
\(\hat{p}_{kNN}\) 0.97 0.90 0.83 0.80 0.97 0.57 0.80 0.61 0.82 0.87 1.10 10
\(\cos_{\alpha }\) 0.94 0.87 0.76 0.79 0.97 0.52 0.78 0.59 0.78 0.83 2.95 7
\(\gamma_{Euc}\) 0.94 0.85 0.77 0.71 0.97 0.51 0.77 0.60 0.77 0.83 3.40 8
\(\kappa_{Euc}\) 0.93 0.85 0.76 0.69 0.96 0.51 0.78 0.59 0.77 0.83 4.05 6
\(\delta_{Euc}\) 0.91 0.82 0.76 0.73 0.94 0.55 0.74 0.58 0.75 0.79 4.70 0
\(\delta_{\text{Tan}}\) 0.90 0.82 0.74 0.68 0.93 0.53 0.72 0.58 0.74 0.79 5.75 0
\(\kappa_{\text{Tan}}\) 0.88 0.80 0.73 0.66 0.91 0.56 0.70 0.57 0.72 0.77 6.90 1
\(\gamma_{\text{Tan}}\) 0.88 0.79 0.73 0.66 0.91 0.55 0.71 0.57 0.72 0.76 7.15 0
Range 0.09 0.11 0.10 0.14 0.06 0.06 0.10 0.04 0.10 0.11   
LDA
\(\hat{p}_{LDA}\) 0.93 0.90 0.77 0.85 0.97 0.54 0.84 0.58 0.76 0.85 1.10 9
\(\cos_{\alpha }\) 0.86 0.87 0.75 0.83 0.97 0.52 0.76 0.57 0.71 0.81 3.55 6
\(\gamma_{Euc}\) 0.86 0.86 0.73 0.79 0.96 0.52 0.79 0.57 0.69 0.82 3.60 6
\(\kappa_{Euc}\) 0.87 0.85 0.72 0.78 0.96 0.52 0.79 0.56 0.69 0.82 3.90 4
\(\delta_{Euc}\) 0.84 0.84 0.75 0.78 0.94 0.54 0.77 0.55 0.69 0.78 4.55 0
\(\delta_{\text{Tan}}\) 0.86 0.83 0.69 0.77 0.93 0.53 0.78 0.55 0.67 0.78 5.55 1
\(\kappa_{\text{Tan}}\) 0.82 0.81 0.70 0.76 0.92 0.53 0.77 0.54 0.67 0.75 6.75 0
\(\gamma_{\text{Tan}}\) 0.82 0.81 0.69 0.76 0.92 0.53 0.77 0.54 0.67 0.74 7.00 0
Range 0.11 0.09 0.08 0.09 0.05 0.02 0.08 0.04 0.09 0.11   
  1. aNumber of data sets where the AD measure performs significantly better than chance based on the 95th percentile (α = 0.05) of the permutation test (see Additional file 1 for a description of the permutation test and Additional file 2 for code of the permutation test)
  2. bUnderlined values indicate that the AD measure performs significantly better than chance based on the permutation test (for details of the permutation test see also footnote a)