Introduce DeepPurpose, a deep learning-based toolkit for drug-target interaction prediction, drug repurposing, and virtual screening.

TL;DR: Deep learning is revolutionizing the drug discovery industry. In this post, I show how to use the DeepPurpose toolkit to unlock 50+ models for Drug-Target Interaction (DTI) prediction, a fundamental task in drug discovery, using only a few lines of code in a scikit-learn fashion. DeepPurpose also enables an easy interface to do two important applications of DTI prediction: virtual screening and drug repurposing.

Introduction

Discover a new drug takes more than 10 years and costs higher than $2.6 billion [1]. Recently, many AI for drug discovery startups emerge and have successfully applied deep learning techniques to aid drug discovery research and greatly shorten time/save cost [2,3]. Thus, it is a very exciting and booming field 💥!

However, the interest within the data scientist community in deep learning for drug discovery, although growing quickly, is still far behind other deep learning applications such as NLP and images. One possible reason is that the lack of open-sourced tools prevents many data scientists to probe into this exciting domain. Existing toolkits such as DeepChem, OpenChem, MoleculeNet and etc., have democratized the use of deep learning for life sciences greatly. In this post, I describe a new addition called DeepPurpose that works specifically for drug-target interaction prediction, a fundamental task for the drug discovery process. All data scientists love scikit-learn. Thus, we design DeepPurpose as a scikit-learn-fashion framework that wraps 50+ models in less than 10 lines 😇!

Background

Let’s start with some biochemistry concepts. A major category of disease is usually attributed to the target protein in the disease pathway. A drug is discovered to modulate the target protein. Through this modulation of disease responsible protein, the disease can be cured. One of the major paradigms of the drug action mechanism is the ‘Lock-And-Key’ theory [4]. A protein is a “ lock” 🔒 and drug discovery is to find the right “key” 🔑 to unlock the target (i.e., the right drug to modulate the protein). This fitness is called binding affinity.

Drug-target interaction (DTI) measures the binding affinity of drug molecules to the protein targets. Thus, we can easily imagine that an accurate DTI deep learning model can greatly benefit the drug discovery process [5]. More specifically, virtual screening and drug repurposing are two main applications based on DTI. Virtual screening helps identify ligand candidates that can bind to the protein of interest, whereas drug repurposing finds new therapeutic purposes for existing drugs.

DeepPurpose Overview

DeepPurpose formulates DTI models as an encoder-decoder framework, which includes a majority of previous works [6,7,8,9,10] and much more. Two encoders generate embeddings for drug and protein respectively and then they are concatenated into a decoder, where it predicts the binding affinity score.

DeepPurpose uses the most accessible input data format. The input is a drug target pair, where the drug uses the simplified molecular-input line-entry system (SMILES) string and the target uses the amino acid sequence. The output is a score indicating the binding activity of the drug-target pair.

For drugs, we provide 8 encoders: four classic cheminformatics fingerprints (Morgan, Pubchem, Daylight, and RDKit 2D) with Deep Neural Network (DNN) on top of them; 1D Convolutional Neural Network (CNN) on SMILES string characters; Recurrent Neural Network on top of CNN to leverage the sequential order; Transformer encoder on sub-structure partition fingerprint [11]; and Message passing neural network on the molecular graphs.

For the target protein, we have 7 encoders: DNN on top of four classic computational biology fingerprints (AAC, PseAAC, Conjoint Triad, Quasi-sequence); CNN on the amino acid sequence; RNN on top of CNN; and transformer on the sub-sequence fingerprint.

In combination, there are 56 (8 times 7) drug-target encoding models! Also, most of the models are novel 🆕!

DeepPurpose Framework

Now after we have a conceptual overview of DTI and DeepPurpose, I begin to introduce the DeepPurpose programming framework. This framework consists of several steps where each step consists of one line of code:

• Data loading
• Encoder specification
• Data encoding and split
• Model configuration generation
• Model initialization
• Model training
• Repurposing/Screening
• Model saving and loading

Let’s go over each step! For a better learning experience, it is recommended to follow these steps in your own Jupyter notebook file or going over the blog post notebook. To install DeepPurpose, you can use the DeepPurpose Binder cloud notebook (simply clicking the link!) or use the local DeepPurpose environment. The instructions for the installation can be found here, where you can also find video installation tutorials.

Data loading

DeepPurpose takes in a NumPy array of drug’s SMILES strings (X_drugs), a NumPy array of target protein’s amino acid sequence (X_targets), and a NumPy array of labels (y), which can either be binary 0/1 indicating interaction outcome or a real number indicating binding affinity value. The input drug and target arrays should be paired, i.e. y[0] is the score for X_drugs[0] and X_targets[0]. DeepPurpose will automatically switch between the binary classification task or regression task depending on the dataset.

Besides transforming into NumPy arrays through some data wrangling on your own, DeepPurpose also provides benchmark dataset loaders (DAVIS/KIBA/BindingDB) to ease preprocessing. For example, in this post, we will use the DAVIS dataset:

You can also load your own dataset from txt file using thedataset.read_file_training_dataset_drug_target_pairs function, where each line is a drug SMILES string, a target amino acid sequence, and the binding score.

Encoder specification

After we obtain the required data format, we need to first specify the encoder to use for drug and protein. Here we try MPNN for the drug and CNN for the target. Note that you can switch encoder just by changing the encoding name. The full list of encoders is listed here.

(If you are using CPU to run this blog post, you will find that MPNN and CNN are a bit large, you can try smaller encoders such as Morgan for drugs and Conjoint_triad for proteins.)

Data encoding and split

Now, we need to prepare the data encoding scheme for different encoders, using the utils.data_process function. In this function, we can specify train/validation/test split fractions, random seed to ensure the same data split for reproducibility, and also supports data splitting methods such as cold_drug and cold_protein, which splits on drug/proteins for model robustness evaluation to test unseen drug/proteins. The function outputs train, validation, and test set Pandas data frames.

Model configuration generation

Now, we initialize a model with its configuration. You can modify almost any hyper-parameters (e.g., learning rate, epoch, batch size), model parameters (e.g. hidden dimensions, filter size) in this function. The supported configurations are listed here in this link.

For this blog post, we specify the epoch size to be 3 so that you can run on both CPUs & GPUs quickly and can proceed to the next steps. For reference parameters, check out the notebooks in the DEMO folder.

Model initialization

Next, we initialize a model using the above configuration:

Model training

Now, it is ready to train by simply typing the model.train function!

The training loss curve will be automatically printed out. Also, if the task is a binary classification task, the test set ROC-AUC and PR-AUC curves will also be printed out.

Repurposing/Screening

After we trained the model, we can simply repurpose and screen using the models.repurpose and models.virtual_screening functions:

For example, suppose we want to do repurposing from a set of antiviral drugs for a COVID-19 target 3CL protease. We have provided the corresponding data in the dataset wrappers.

A ranked list of drug candidates are automatically generated and printed out:

Next, we showcase how to do virtual screening on a sample of data from the BindingDB dataset and then use the virtual_screening function to generate a list of drug-target pairs that have high binding affinities. If no drug/target names are provided, the index of the drug/target list is used instead. A similar looking ranked list would be generated.

Model saving and loading

At last, saving and loading models are also really easy. The loading function also automatically detects if the model is trained on multiple GPUs. An example to save and load the model we just trained:

We have also provided a list of pretrained model, you can find all available ones under the list. For example, to load an MPNN+CNN model pretrained on BindingDB Kd dataset:

That’s it! You can now train a state-of-the-art deep learning model for the drug-target interaction prediction task 👏!

DeepPurpose also supports many more functionalities, for example, this demo shows how to use the Ax platform to do the latest hyperparameter tuning methods such as Bayesian Optimization on DeepPurpose.

Last but not least, many real-world drug development involves assay data, which means the data only consists of a drug molecule and the binding score to the target or bacteria. This problem is also called a drug property prediction task. DeepPurpose also supports this task by using only the drug encoders to predict the binding score. The framework is very similar to DTI prediction, as illustrated here:

For more detailed step-by-step instruction, you can find the notebook for this blog post here: [DTI Prediction][Drug Property Prediction].

This wraps up our blog post on DeepPurpose and DTI prediction! I hope you like it and become interested in the exciting intersection of deep learning and drug discovery if you haven’t already! We also call for data scientists to contribute to this open-sourced project!

To learn more, here is the paper and Github repository, please star, share, and contribute!

This is joint work with Tianfan Fu, Lucas Glass, Marinka Zitnik, Cao Xiao, and Jimeng Sun.

If you have any questions or feedback, feel free to reach me at kexinhuang@hsph.harvard.edu; or simply PM me on Twitter @KexinHuang5!

Reference

[1] Mullard, A. New drugs cost US$2.6 billion to develop. Nature Reviews Drug Discovery (2014).

[2] Fleming, Nic. How artificial intelligence is changing drug discovery. Nature (2018).

[3] Smalley, E. AI-powered drug discovery captures pharma interest. Nature Biotechnology (2017).

[4] Gschwend DA, Good AC, Kuntz ID. Molecular docking towards drug discovery. Journal of Molecular Recognition: An Interdisciplinary Journal (1996).

[5] Mayr, Andreas, et al. Large-scale comparison of machine learning methods for drug target prediction on ChEMBL. Chemical science (2018).

[6] Öztürk H, Özgür A, Ozkirimli E. DeepDTA: deep drug-target binding affinity prediction. Bioinformatics (2018).

[7] Nguyen, Thin, Hang Le, and Svetha Venkatesh. GraphDTA: prediction of drug–target binding affinity using graph convolutional networks. BioRxiv (2019).

[8] Tsubaki, Masashi, Kentaro Tomii, and Jun Sese. Compound–protein interaction prediction with end-to-end learning of neural networks for graphs and sequences. Bioinformatics (2019).

[9] Lee, Ingoo, Jongsoo Keum, and Hojung Nam. DeepConv-DTI: Prediction of drug-target interactions via deep learning with convolution on protein sequences. PLoS computational biology (2019).

[10] Chen, Xing, et al. Drug–target interaction prediction: databases, web servers and computational models. Briefings in bioinformatics (2016).

[11] Huang K, Xiao C, Hoang T, Glass L, Sun J. CASTER: Predicting Drug Interactions with Chemical Substructure Representation. AAAI (2020).

Drug Discovery with Deep Learning Under 10 Lines of Codes. was originally published in TDS Archive on Medium, where people are continuing the conversation by highlighting and responding to this story.