Table 5 Performance evaluation metrics of the optimal ML model and DL model with different activity cutoff values
Cutoff value | Model | ACC | F1 | AUC |
|---|---|---|---|---|
|  | ABDT based on MACCS | 92.28 ± 0.00 | 93.02 ± 0.00 | 93.94 ± 0.00 |
0.5 μM | DL based on MACCS + ECFP4 (rank 1) | 88.05 ± 1.78 | 88.68 ± 1.98 | 94.87 ± 0.62 |
|  | DL based on physicochemical descriptors + MACCS (rank 2) | 87.14 ± 1.93 | 87.93 ± 2.15 | 94.69 ± 0.89 |
|  | DL based on SMILES | 90.40 ± 0.94 | 91.47 ± 0.93 | 95.20 ± 0.43 |
|  | GBDT based on physicochemical descriptors + ECFP4 + MACCS | 87.68 ± 0.09 | 88.88 ± 0.10 | 91.59 ± 0.16 |
1 μM | DL based on physicochemical descriptors + ECFP4 + MACCS (rank 1) | 82.13 ± 3.54 | 82.83 ± 4.08 | 93.92 ± 1.40 |
|  | DL based on MACCS + ECFP4 (rank 2) | 83.62 ± 3.54 | 84.44 ± 3.95 | 93.89 ± 1.43 |
|  | DL based on SMILES | 88.48 ± 0.88 | 89.92 ± 0.94 | 93.21 ± 0.54 |
|  | RF based on ECFP4 and SMOTE | 84.94 ± 1.25 | 90.04 ± 0.84 | 90.20 ± 0.97 |
10 μM | DL based on physicochemical descriptors + MACCS and SMOTE (rank 1) | 75.91 ± 2.99 | 81.68 ± 2.73 | 84.32 ± 3.17 |
|  | DL based on ECFP4 + MACCS and SMOTE (rank 2) | 76.98 ± 3.84 | 83.23 ± 3.35 | 84.00 ± 2.96 |
|  | DL based on MACCS and SMOTE (rank 4) | 74.80 ± 2.40 | 80.80 ± 1.96 | 83.70 ± 3.43 |
|  | DL based on physicochemical descriptors + ECFP4 and unbiased decoy selection (rank 5) | 76.90 ± 1.29 | 82.45 ± 1.18 | 83.09 ± 1.37 |
|  | DL based on SMILES and SMILES enumeration | 74.02 ± 3.08 | 81.34 ± 1.93 | 79.47 ± 2.07 |