Deep learning has been applied to solve complex problems in a variety of scientific domains including biological and medical sciences. In particular, convolutional neural networks (CNNs) have displayed state-of-the-art results in computer vision tasks, outperforming humans in numerous cases. In this work, we explore several deep learning applications in medicine and bioinformatics. We show that deep learning can be applied to real-world medical procedures such as colon screening, and assist physicians with polyp detection and segmentation, and that deep learning can be applied to optical surgery, and assist physicians with image quality enhancement. In the field of bioinformatics, advances in high-throughput sequencing technologies have greatly lowered the cost of genome-wide sequencing, allowing researchers to include a larger number of experimental replicates than they previous could. This allows an easier application of deep learning as well as other statistical models to solve the problems in bioinformatics, such as detecting circadian rhythm in high-throughput omic data. To access and mine circadian datasets in a comprehensive and integrated way, we curate over 227 high-thoughput circadian datasets across different species and tissues, apply a deep learning model named BIO_CYCLE together with other statistical methods to detect circadian pattern in the omic data, and build a web portal http://circadiomics.ics.uci.edu for search and visualization.

One of the widely discussed problems in the feld of chemoinformatics is the prediction of molecular properties. These properties can range from physical, chemical, or biological properties of molecules to the behaviour or molecules under certain chemical conditions.Traditionally, these properties were calculated using chemical experiments. But with the increase in computational capabilities, various machine learning methods like neural networks and kernel methods have also been tried. These approaches have been successful to a certain extent. But with recent development in data, deep machine learning techniques have been developed, which matches or exceed the performance of state-of-the-art techniques. One such approach is to consider the molecular graphs of the molecules and use them for prediction. Here, I discuss this approach in great details and provide its application on two problems: predicting aqueous solubility and melting point.

Weight-sharing is one of the pillars behind Convolutional Neural Networks and their successes. However, in physical neural systems such as the brain, weight-sharing is implausible. Taking inspiration from neural circuitry, we explore the use of Free Convolutional Networks and neurons with variable connection patterns. Using Free Convolutional Networks, we show that while weight-sharing is a pragmatic optimization approach, it is not a necessity in computer vision applications.

Second, we turn to an application of artificial networks for scientific computing. Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge in order to connect environments where deep learning resources are plentiful with those where they are scarce.

Finally, we examine an application of neural networks to activity classification. Using raw EEG signals we demonstrate superior performance of our novel attention based network to predict a subject's motor execution.

The side-chain prediction and residue-residue contact prediction are sub-problems in the protein structure prediction. Both predictions are important for protein prediction and other applications.

We have developed a new algorithm, SIDEpro, for the side-chain prediction where an energy function for each rotamer in a structure is computed additively over pairs of contacting atoms. A family of 156 neural networks indexed by amino acids and contacting atom types is used to compute these rotamer energies as a function of atomic contact distances. Although direct energy targets are not available for training, the neural networks can still be optimized by converting the energies to probabilities and optimizing these probabilities using Markov Chain Monte Carlo methods. The resulting predictor SIDEpro makes predictions by initially setting the rotamer probabilities for each residue from a backbone-dependent rotamer library, then iteratively updating these probabilities using the trained neural networks. After convergences of the probabilities, the side-chains are set to the highest probability rotamer. Finally, a post processing clash reduction step is applied to the models. SIDEpro represents a significant improvement in speed and a modest, but statistically significant, improvement in accuracy when compared with the state-of-the-art for rapid side-chain prediction method SCWRL4 on the 379 protein test set of SCWRL4. Using the SCWRL4 test set, SIDEpro's accuracy (χ₁ 86.14%, χ₁₊₂ 74.15%) is slightly better than SCWRL4-FRM (χ₁ 85.43%, χ₁₊₂ 73.47%) and it is 7.0 times faster. SIDEpro can also predict the side chains of proteins containing non-standard amino acids, including 15 of the most frequently observed PTMs in the Protein Data Bank and all types of phosphorylation. For PTMs, the χ₁ and χ₁₊₂ accuracies are comparable with those obtained for the precursor amino acid, and so are the RMSD values for the atoms shared with the precursor amino acid. In addition, SIDEpro can accommodate any PTM or unnatural amino acid, thus providing a flexible prediction system for high-throughput modeling of proteins beyond the standard amino acids.

We have also developed a novel machine learning approach for contact map prediction using three steps of increasing resolution. First, we use 2D recursive neural networks to predict coarse contacts and orientations between secondary structure elements. Second, we use an energy-based method to align secondary structure elements and predict contact probabilities between residues in contacting alpha-helices or strands. Third, we use a deep neural network architecture to organize and progressively refine the prediction of contacts, integrating information over both space and time. We train the architecture on a large set of non-redundant proteins and test it on a large set of non-homologous domains, as well as on the set of protein domains used for contact prediction in the two most recent CASP8 and CASP9 experiments. For long-range contacts, the accuracy of the new CMAPpro predictor is close to 30%, a significant increase over existing approaches.

Both SIDEpro and CMAPpro are part of the SCRATCH suite of predictors and available from: http://scratch.proteomics.ics.uci.edu/.

In the statistics and machine learning communities, there exists a perceived dichotomy be- tween statistical inference and out-of-sample prediction. Statistical inference is often done with models that are carefully specified a priori while out-of-sample prediction is often done with “black-box” models that have greater flexibility. The former is more concerned with model theoretical properties when data become infinite; the later focuses more on algorithms that scale up to larger data sets. To a scientist who is outside of these communities, the distinction of inference and prediction might not seem so clear. With technological advancements, scientists can now collect overwhelming amounts of data in various formats and their objective is to make sense of the data. To this end, we propose the synergy of statistical inference and prediction workhorses that are neural networks and Gaussian processes. Despite hardware improvements under Moore’s law, ever bigger data and more complex models pose computational challenges for statistical inference. To address these computational challenges, we approximate functional forms of the data to effectively reduce the burden of model evaluation. In addition, we present a case study where we use flexible models to learn scientifically interesting representations of rat memories from experimental data for better understanding of the brain.

Progress in the sciences depends critically on the analysis of ever-growing bodies of data. Many of these analysis patterns are inferential in nature; their goal is to infer the value of one or more parameters which bear some real-world meaning. Others are in essence discriminative; their goal is to build a black-box model with the strongest possible predictive power. For both of these analysis styles, machine learning offers a host of powerful tools to tackle historically unapproachable problems.

In this dissertation, I present three examples of machine learning tools applied to the sciences. The first offers a novel model of textual diversity applied to the science of science itself. The second, explores a series of discriminative models which probe the evolution of the cosmos. The third offers a novel convolutional neural architecture for discriminating effective from ineffective drug candidates.

Taken together, these studies offer a glimpse of the breadth and potency of the contributions that machine learning can offer to the sciences.

There are numerous applications for deep learning in a healthcare setting including: providing more accurate diagnoses, recommending treatment plans, predicting patient outcomes, tracking patient engagement and adherence, and reducing the burden of administrative tasks. This plethora of applications has resulted in the widespread publication of deep learning algorithms applied to healthcare data. Despite numerous publications showing deep learning to be very successful in retrospective healthcare studies, very few of these algorithms are then actually incorporated into clinical practice. While there are many factors influencing the lack of algorithm deployment, one of the major reasons is a lack of trust in deep learning. This lack of trust stems in part from a lack of model interpretability and an inability to independently verify published results due to a lack of data availability. In this work, we explore generalized additive models with neural networks (GAM-NNs) as a method of improving model interpretability and we propose MOVER: Medical Informatics Operating Room Vi- tals and Events Repository a publicly available repository of medical data designed to improve visibility into deep learning algorithms in healthcare.

Similarly, deep learning can be used to analyze circadian omic (e.g. transcriptomic, metabolomic, proteomic) time series data. Several studies have shown that a disruption to circadian rhythms have been linked to health problems such as cancer, diabetes, obesity, and premature aging. In order to gain clinician trust in the conclusions drawn from circadian omic analyses we propose CircadiOmics: the largest annotated repository of circadian omic time series data analyzed using deep learning. Clinicians and researchers can use CircadiOmics to not only validate the findings of their circadian omic experiments, but also to analyze multiple circadian omic experiments in aggregate.