The power metric: a new statistically robust enrichment-type metric for virtual screening applications with early recovery capability

Lopes, Julio Cesar Dias; dos Santos, Fábio Mendes; Martins-José, Andrelly; Augustyns, Koen; De Winter, Hans

doi:10.1186/s13321-016-0189-4

Table 1 Dependency on the model quality parameter λ using models generated from datasets with 100 actives (n) on 10,000 compounds in total (N)

From: The power metric: a new statistically robust enrichment-type metric for virtual screening applications with early recovery capability

Metric	λ					χ (%)
Metric	2	5	10	20	40	χ (%)
PM	0.51 ± 0.35	0.74 ± 0.24	0.89 ± 0.09	0.95 ± 0.02	0.98 ± 0.01	0.5
ROCE	2.35 ± 2.18	5.13 ± 3.39	10.46 ± 4.99	22.34 ± 7.86	49.96 ± 14.35
EF	2.28 ± 2.06	4.83 ± 3.03	9.38 ± 4.04	18.08 ± 5.17	32.94 ± 6.22
REF	2.28 ± 2.06	4.83 ± 3.03	9.38 ± 4.04	18.08 ± 5.17	32.94 ± 6.22
CCR	0.50 ± 0.01	0.51 ± 0.01	0.52 ± 0.01	0.54 ± 0.01	0.58 ± 0.02
MCC	0.01 ± 0.01	0.03 ± 0.02	0.06 ± 0.03	0.12 ± 0.04	0.23 ± 0.04
CKC	0.01 ± 0.01	0.03 ± 0.02	0.06 ± 0.03	0.11 ± 0.03	0.21 ± 0.04
SEN	0.01 ± 0.01	0.02 ± 0.02	0.05 ± 0.02	0.09 ± 0.03	0.16 ± 0.03
SPE	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00
PRE	0.02 ± 0.02	0.05 ± 0.03	0.09 ± 0.04	0.18 ± 0.05	0.33 ± 0.06
ACC	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00
PM	0.61 ± 0.23	0.80 ± 0.11	0.90 ± 0.04	0.95 ± 0.01	0.98 ± 0.00	1
ROCE	2.32 ± 1.55	5.07 ± 2.31	10.19 ± 3.34	20.97 ± 5.08	44.00 ± 8.22
EF	2.26 ± 1.48	4.83 ± 2.09	9.25 ± 2.75	17.33 ± 3.46	30.54 ± 3.95
REF	2.26 ± 1.48	4.83 ± 2.09	9.25 ± 2.75	17.33 ± 3.46	30.54 ± 3.95
CCR	0.51 ± 0.01	0.52 ± 0.01	0.54 ± 0.01	0.58 ± 0.02	0.65 ± 0.02
MCC	0.01 ± 0.02	0.04 ± 0.02	0.08 ± 0.03	0.17 ± 0.03	0.30 ± 0.04
CKC	0.01 ± 0.02	0.04 ± 0.02	0.08 ± 0.03	0.17 ± 0.03	0.30 ± 0.04
SEN	0.02 ± 0.01	0.05 ± 0.02	0.09 ± 0.03	0.17 ± 0.03	0.31 ± 0.04
SPE	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00	0.99 ± 0.00
PRE	0.02 ± 0.01	0.05 ± 0.02	0.09 ± 0.03	0.17 ± 0.03	0.31 ± 0.04
ACC	0.98 ± 0.00	0.98 ± 0.00	0.98 ± 0.00	0.98 ± 0.00	0.99 ± 0.00
PM	0.66 ± 0.13	0.82 ± 0.06	0.90 ± 0.02	0.95 ± 0.01	0.97 ± 0.00	2
ROCE	2.30 ± 1.08	4.91 ± 1.56	9.69 ± 2.18	18.75 ± 3.08	35.21 ± 4.06
EF	2.26 ± 1.03	4.70 ± 1.43	8.88 ± 1.82	15.87 ± 2.19	26.17 ± 2.23
REF	4.52 ± 2.07	9.40 ± 2.85	17.76 ± 3.65	31.74 ± 4.38	52.34 ± 4.45
CCR	0.51 ± 0.01	0.54 ± 0.01	0.58 ± 0.02	0.65 ± 0.02	0.75 ± 0.02
MCC	0.02 ± 0.01	0.05 ± 0.02	0.11 ± 0.03	0.21 ± 0.03	0.36 ± 0.03
CKC	0.02 ± 0.01	0.05 ± 0.02	0.11 ± 0.02	0.20 ± 0.03	0.34 ± 0.03
SEN	0.05 ± 0.02	0.09 ± 0.03	0.18 ± 0.04	0.32 ± 0.04	0.52 ± 0.04
SPE	0.98 ± 0.00	0.98 ± 0.00	0.98 ± 0.00	0.98 ± 0.00	0.99 ± 0.00
PRE	0.02 ± 0.01	0.05 ± 0.01	0.09 ± 0.02	0.16 ± 0.02	0.26 ± 0.02
ACC	0.97 ± 0.00	0.97 ± 0.00	0.97 ± 0.00	0.98 ± 0.00	0.98 ± 0.00

Metric abbreviations are given in the Methods section. All metrics are dependent on the model quality, but in case of the ROCE, EF, REF, MCC, CKC, SEN and PRE metrics there is at least a tenfold increase when moving from a bad model (λ = 2) to a good model (λ = 40), while for the PM metric there is a doubling of the value. The accuracy ACC and specificity SPE metrics are not dependent on the quality of model, while the correct classification rate metric (CCR) shifts from 0.5 in the case of a bad model to a maximum of 0.75 for the best model. Good models have a PM of >0.9; for good models this value is largely independent on the applied cutoff value χ (see Table 3 as well)

Back to article page

ISSN: 1758-2946

Contact us

Submission enquiries: journalsubmissions@springernature.com

Journal of Cheminformatics

Contact us