Skip to main content

Table 2 Accuracy metrics for agonist activity evaluation with different numbers of activity sources per chemical

From: Exploring non-linear distance metrics in the structure–activity space: QSAR models for human estrogen receptor

# sources # chemicals Model and parameters Sensitivity Specificity Bal accuracy Accuracy ROC AUC Score
1 6197 CERAPP consensus 0.71 0.95 0.83 0.94 0.85 0.67
1 6197 Morgan kNN arithm k = 10 0.55 0.96 0.75 0.94 0.82 0.58
1 6197 Morgan kNN geom k = 2 0.38 0.99 0.69 0.97 0.72 0.48
1 6197 Morgan kNN exp k = 10 X = 1.5 0.59 0.97 0.78 0.95 0.83 0.61
1 6197 Morgan GkNN k = 10 X = 1 Y = 1 0.58 0.96 0.77 0.94 0.82 0.59
1 6197 Morgan GkNN k = 10 X = 1 Y = 3 0.59 0.97 0.78 0.95 0.83 0.61
1 6197 Morgan GkNN k = 10 X = 1.5 Y = 3 0.64 0.93 0.78 0.92 0.82 0.59
1 6197 Morgan GkNN k = 20 X = 1.5 Y = 5 0.64 0.94 0.79 0.93 0.83 0.61
3 1553 CERAPP consensus 0.93 0.94 0.94 0.94 0.98 0.87
3 1553 Morgan kNN arithm k = 10 0.77 0.95 0.86 0.94 0.94 0.76
3 1553 Morgan kNN geom k = 2 0.57 0.99 0.78 0.97 0.80 0.60
3 1553 Morgan kNN exp k = 10 X = 1.5 0.82 0.97 0.89 0.96 0.95 0.81
3 1553 Morgan GkNN k = 10 X = 1 Y = 1 0.82 0.96 0.89 0.95 0.94 0.79
3 1553 Morgan GkNN k = 10 X = 1 Y = 3 0.83 0.97 0.90 0.96 0.95 0.82
3 1553 Morgan GkNN k = 10 X = 1.5 Y = 3 0.88 0.93 0.90 0.92 0.95 0.79
3 1553 Morgan GkNN k = 20 X = 1.5 Y = 5 0.88 0.94 0.91 0.94 0.94 0.80
5 456 CERAPP consensus 0.96 0.93 0.94 0.94 0.99 0.88
5 456 Morgan kNN arithm k = 10 0.81 0.94 0.88 0.93 0.94 0.77
5 456 Morgan kNN geom k = 2 0.68 1.00 0.84 0.96 0.86 0.69
5 456 Morgan kNN exp k = 10 X = 1.5 0.92 0.97 0.94 0.96 0.96 0.87
5 456 Morgan GkNN k = 10 X = 1 Y = 1 0.89 0.95 0.92 0.94 0.95 0.82
5 456 Morgan GkNN k = 10 X = 1 Y = 3 0.92 0.97 0.94 0.96 0.96 0.87
5 456 Morgan GkNN k = 10 X = 1.5 Y = 3 0.94 0.92 0.93 0.92 0.96 0.82
5 456 Morgan GkNN k = 20 X = 1.5 Y = 5 0.94 0.95 0.94 0.95 0.96 0.86
7 128 CERAPP consensus 0.95 0.95 0.95 0.95 1.00 0.90
7 128 Morgan kNN arithm k = 10 0.88 0.98 0.93 0.95 0.95 0.84
7 128 Morgan kNN geom k = 2 0.76 1.00 0.88 0.94 0.90 0.74
7 128 Morgan kNN exp k = 10 X = 1.5 0.91 0.99 0.95 0.97 0.96 0.89
7 128 Morgan GkNN k = 10 X = 1 Y = 1 0.94 0.99 0.97 0.98 0.96 0.90
7 128 Morgan GkNN k = 10 X = 1 Y = 3 0.94 1.00 0.97 0.98 0.97 0.92
7 128 Morgan GkNN k = 10 X = 1.5 Y = 3 0.94 0.91 0.93 0.92 0.96 0.82
7 128 Morgan GkNN k = 20 X = 1.5 Y = 5 0.94 0.97 0.95 0.96 0.97 0.89
9 57 CERAPP consensus 0.92 1.00 0.96 0.97 1.00 0.93
9 57 Morgan kNN arithm k = 10 0.79 1.00 0.89 0.93 0.93 0.78
9 57 Morgan kNN geom k = 2 0.79 1.00 0.89 0.93 0.92 0.77
9 57 Morgan kNN exp k = 10 X = 1.5 0.84 1.00 0.92 0.95 0.94 0.82
9 57 Morgan GkNN k = 10 X = 1 Y = 1 0.84 1.00 0.92 0.95 0.94 0.82
9 57 Morgan GkNN k = 10 X = 1 Y = 3 0.84 1.00 0.92 0.95 0.94 0.82
9 57 Morgan GkNN k = 10 X = 1.5 Y = 3 0.89 0.92 0.91 0.91 0.94 0.78
9 57 Morgan GkNN k = 20 X = 1.5 Y = 5 0.89 0.97 0.93 0.95 0.94 0.84
  1. “kNN arithm”, “kNN geom”, and “kNN exp” indicate the kNN models with the arithmetic, geometric, and exponential averaging, respectively. The cumulative score shown in the last column is the product of balanced accuracy, accuracy, and ROC AUC. Italic font indicates accuracy metric values that exceed those for the CERAPP consensus model