ASFP (AI-based Scoring Function Platform): a web server for the development of customized scoring functions

Virtual screening (VS) based on molecular docking has emerged as one of the mainstream technologies of drug discovery due to its low cost and high eciency. However, the scoring functions (SFs) implemented in most docking programs are not always accurate enough and how to improve their prediction accuracy is still a big challenge. Here, we propose an integrated platform called ASFP, a web server for the development of customized SFs for structure-based VS. There are three main modules in ASFP: 1) the descriptor generation module that can generate up to 3437 descriptors for the modelling of protein-ligand interactions; 2) the AI-based SF construction module that can establish target-specic SFs based on the pre-generated descriptors through three machine learning (ML) techniques; 3) the online prediction module that provides some well-constructed target-specic SFs for VS and a generic SF for binding anity prediction. Our methodology has been validated on several benchmark datasets. The target-specic SFs can achieve an average ROC AUC of 0.841 towards 32 targets and the generic SF can achieve the Pearson correlation coecient of 0.81 on the PDBbind version 2016 core set. To sum up, the ASFP server is a powerful tool for structure-based VS and binding anity prediction. Availability and Implementation: ASFP web server is freely available at http://cadd.zju.edu.cn/asfp/.


Introduction
As one of the core technologies in virtual screening (VS), molecular docking has been extensively used to screen small molecule libraries for lead discovery. 1 A protein-ligand docking algorithm consists of two basic components: a search algorithm to generate a large number of potential ligand binding poses within the binding site and a scoring function (SF) to evaluate the binding strength for a particular pose.
In general, most SFs implemented in docking programs cannot give a reliable prediction to the relative binding a nity of a set of compounds. 2 Therefore, how to improve the accuracy of SFs still remains a big challenge.
Traditional SFs can be roughly classi ed into three categories:1) force eld-based SFs, 2) knowledgebased SFs and 3) empirical SFs. Unlike traditional SFs, MLSFs do not have particular theory-motivated functional forms, and they are developed by learning from very large volumes of protein-ligand structural and interaction data through ML algorithms, such as random forest (RF), support vector machine (SVM), arti cial neural network (ANN), gradient boosting decision tree (GBDT), etc. [3][4][5][6][7] Consequently, MLSFs have the capability to capture the non-linear relationship between protein-ligand interaction features and binding a nities that are di cult to be characterized by classical SFs, thus yielding better binding a nity predictions. However, in order to develop a MLSF, we need to generate a set of features to characterize protein-ligand interactions, and furthermore we need to be familiar with ML algorithms, which may be a di cult task for non-experts.
Here, we developed the ASFP server that can be used to develop customized MLSFs for structure-based VS and provide a generic MLSF for binding a nity prediction. The ASFP server has three basic modules: descriptor generation, AI-based SF construction and online prediction. In the descriptor generation module, 15 computational tools (only 9 tools are available due to license restriction) are embedded into the module for the characterization of ligand, protein binding pocket and protein-ligand interaction information, and up to 3437 descriptors can be generated. The AI-based SF construction module can be used to develop customized SFs with easy operation. In the online prediction module, 15 well-validated target-speci c classi cation models for VS and a generic regression model for binding a nity prediction are provided for users. All the above modules in the ASFP server are automated and the results are presented interactively through a user-friendly interface.

Implementation
The implementation of ASFP consists of two parts: the model construction and validation and the development of the web server that purposes in ML-based SF construction.

Model construction
Benchmark. The benchmark dataset I (Dataset I), which contains the kinase subset and the diverse subset in the Directory of Useful Decoys-Enhanced (DUD-E) benchmark, was used to train and assess the MLSFs. The kinase subset contains the inhibitors and decoys generated by DUDE for 26 kinases, and the diverse subset contains the inhibitors and decoys for seven representative targets in the entire DUDE set.
The basic information of Dataset I is shown in Table S1.
The benchmark dataset (Dataset ) extracted from the PDBbind database (version 2016) 8 was used to train and evaluate the SVM regression model for binding a nity prediction. There are 4057 protein-ligand complexes in the "re ned set" and 290 complexes in the "core set" of PDBbind version 2016.
Evaluation criteria. In this study, six evaluation criteria were utilized to assess the performance of the models. Among them, F1 score, Cohen's kappa, Matthews correlation coe cient (MCC), the area under the receiver operating characteristic curve (ROC AUC) and the enrichment factor (EF) at 1% were used to evaluate the performance of target-speci c models while the Pearson correlation coe cient (R p ) was calculated to assess the performance of the SVM regression model. The details of the metrics can be found in Supplementary material.
Preparation. The protein targets were prepared by using the Structure Preparation wizard in Schrodinger version 2018, which added hydrogen atoms, repaired the side-chains of the imperfect residues using Prime, and optimized the steric hindrance of side-chains. The protonation states of the proteins were determined by using PROPKA and the het groups were preprocessed by Epik. The ligands were prepared using the ligprep module, which added hydrogen atoms, ionized the structures using Epik, desalted, generated tautomers and stereoisomers. In the preparation process, the default settings were used.
Docking. The grids were rstly generated by using the Receptor Grid Generation utility with the size of binding box set to 10 Å × 10 Å × 10 Å centered on the co-crystallized ligand. Then, the Glide docking program with the SP scoring mode was used to dock the prepared ligands into the prepared proteins. For every ligand, only the pose with the highest docking score will be retained.
Descriptors generation. After molecular docking, the structural les of Dataset I and Dataset were retained for descriptors generation. In this study, a total of 15 descriptors calculation tools of various types were included in computing descriptors (Table 1). Considering some of the tools were restricted by license, two schemes were employed to generate the descriptors to establish MLSFs. First, all the SFs (excluding ngerprints and dpocket) supported by the computational tools in Table 1 were used to generate descriptors (ALL descriptors). Second, all the SFs supported by the computational tools without licenses restrictions in Table 1 (i.e. A Score version 3.0, AutoDock version 6.8, DSX version 0.9, GalaxyDockBP2, NNScore version 2.01 and SMoG2016) were used to generate descriptors (FREE descriptors). Both descriptors were implemented in the generic SF construction while only FREE descriptors were utilized to build target-speci c classi cation models due to the huge computational cost. methods. These energy components correlated with the binding a nity of protein-ligand complexes can be used as the input for the development of MLSFs. Therefore, 12 scoring programs were integrated into this module and the scoring components from the output of the SFs implemented in these computational tools can be generated automatically. Besides, two computational tools, i.e., RDkit and PaDEL, were integrated into this module to calculate the Extended-connectivity ngerprint (ECFP) and Pubchem ngerprint, respectively, to characterize the structural features of small molecules. Furthermore, the function of fpocket was supported by this module to calculate 49 descriptors to characterize the structural information of protein pockets. It should be noted that the protein-ligand complexes should be docked before submitted to the server and the descriptors for small molecules may not be recommended for the development of MLSFs. The information of the 15 computational tools supported by ASFP are listed in Table 1. Because some computational tools implemented by ASFP are commercial, and therefore their functions are disabled. Based on the descriptors generated by this module, users can further construct a customized SF through a ML algorithm.
AI-Based Scoring Functions Construction. As one of the modules implemented in the server, the AI-based SF construction is designed for building customizing target-speci c MLSFs. After submission, the work ow is summarized in Fig. 1. In this module, the 384 descriptors computed and extracted from the SFs implemented in 6 freely available computational tools (A Score version 3.0, AutoDock version 6.8, DSX version 0.9, GalaxyDockBP2, NNScore version 2.01 and SMoG2016) can be used for training SFs.
First, the whole dataset uploaded by the user is divided into the training set and the test set according to the user's input. Then, the dataset is preprocessed (standardization, removing features with low variance, and tree-based feature selection) using sklearn. For the sake of computational e ciency, three popular ML algorithms (RF, SVM and XGBoost) are provided. Users can choose a ML algorithm for training and set some options about hyperparameter optimization (which hyperparameter to be optimized, the hyperparameter range and the optimization times). Finally, according to the user's input, the server uses hyperopt to nd the optimal hyperparameter combinations and chooses the corresponding ML algorithm for training (10-fold cross validation) and prediction, and then outputs the results with a PDF le.
Online Prediction. On the base of the model performance, 15 well-constructed customized SFs with research-worthy targets and the generic regression SF for binding a nity prediction were retained to form the third module, Online prediction. The detailed information of the models is provided in Table 2. The ASFP server based on a high-level Python web framework of Django is deployed on a Linux server of an Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20 GHz CPUs with 28 cores and 64 GB of memory. Several SFs programs like autodock 9 were integrated to automate the calculation process. The overall work ow implemented in the ASFP server is shown in Supplementary Figure S1, and the manual of ASFP can be downloaded from the website (http://cadd.zju.edu.cn/asfp/).

Results
As is shown in the Fig. 2A and 2B, target-speci c SFs constructed by ASFP outperformed the docking method, Glide SP, achieving an average ROC AUC of 0.841 towards 32 targets on the DUDE dataset. As for binding a nity prediction, the generic SF can achieve the Pearson correlation coe cient of 0.81 on the PDBbind version 2016 core set 8 , which is comparable to the state-of-the-art regression MLSFs (Fig. 2C). The average speed of modeling is 10 ligand per minute which is in uenced by the ligand size and the computational capacities.

Discussion
All the three modules of ASFP required protein and ligand les uploaded and users can not only get satisfactory results as described in this paper by easily click the 'Run' button using default settings but also be allowed to submit jobs with their own settings. As shown above, the ML-based SFs constructed by ASFP outperform the classic SF (Glide sp) and can be built easily through the ASFP server. Therefore, our ASFP server is a powerful tool that can calculate descriptors for modeling and construct ML-based SFs for virtual screening and binding a nity prediction.
To illustrate the practicability of the ASFP server, if one wants to construct an ML-based SF to nd ligands targeting at Tyrosine-protein kinase ABL (abl1), one can use the AI-Based Scoring Functions Construction module with the input les including a ligand le in the MOL2 format containing 50 active molecules, a decoy le in the MOL2 format containing 150 molecules, a test le in the MOL2 format containing 100 molecules and a protein le in the PDB format (PDB ID: 2HZI10). Upload the les and submit the job with the default hyperparameters settings. As shown in Figure 3, the ASFP server succeeds in generating descriptors and constructing a customized MLSF. The returned PDF le shows that the SF successfully identi es 23 inhibitors from 100 molecules (25 inhibitors).

Conclusions
Here, we present a user-friendly ASFP server for customizing SFs for structure-based VS. We have validated our methodology on several benchmark datasets, and the target-speci c SFs constructed by ASFP achieved an average ROC AUC of 0.841 towards 32 targets on the DUDE dataset and the generic SF can achieve the Pearson correlation coe cient of 0.81 on the PDBbind version 2016 core set, suggesting that the ASFP server is a useful and effective tool for MLSF construction. The combination of 15 computational descriptor generation tools, sklearn and hyperopt makes it very convenient to calculate different types of descriptors and construct customized MLSFs. The ASFP server is an on-going project and further developments will be focused on the integration of more descriptor generation tools, the development of an automatic modelling pipeline using deep learning algorithms (e.g. 3D-convolutional neural networks) and the acceleration in computational speed with the help of more computing resources.

Declarations
Availability and requirements