Entropy is a fundamental concept in science. It describes the disorder, randomness, and uncertainty of a physical, biological, or social system. While understanding entropy has far-reaching impact to advancing our knowledge in many scientific areas and our society, the development of rigorous theories and computational technologies for entropy is a rather challenging task due to the vast complexity of an underlying system. In the context of biological molecules such as proteins and DNAs, entropy as defined in statistical mechanics and thermodynamics is a critical part of the total free energy of such molecules in a chemical environment. Efficient and accurate calculations of such entropy is of particular interest as the calculation of free energy, which is fundamental to physical and biological processes, is known to be notoriously difficult. The need and recent interest in advanced computational methods for entropy in biological molecular systems have motivated directly this dissertation work.

The basic mathematical and statistical definition of entropy, the Shannon entropy in information theory, for a random variable in an Euclidean space is the negative expectation of the natural logarithm of the probability density function (PDF) for the random variable. The entropy of a physical and biological system can be written in the form of, or approximated by, the Shannon entropy with a suitably defined PDF that has physical meanings. Practically, the dimension of an underlying random variable can be very high, and in addition, its PDF may not be known. The goal of my study is to develop efficient and accurate computational methods for the Shannon entropy with the application to the calculation of entropy of a particle system that may consist of many particles, forming a liquid.

In this dissertation work, I begin with a formal derivation of a class of nonparametric kNN-type estimators of the entropy, including the classical kNN estimator, the kpN estimator introduced recently by some physicists, and a new estimator called kp-kernel estimator that I have constructed. One of my objectives here is to understand if theses estimators can better capture some properties that are related to singular behaviors of an underlying PDF, such as the ``tail'' part of the PDF. My extensive numerical simulations using these estimators with several different PDFs show some of such advanced features. These include a better description of a strongly correlated system and more accurate sampling of the tail part of a given distribution. I will then present a convergence analysis to show that some of these estimators converge in expectation, under some realistic assumptions.

Subsequently, I apply these kNN-type entropy estimators to calculate the entropy of simple molecular systems. Here a statistical mechanics theory of simple liquids is invoked, and the entropy is expressed in a series of terms, each is a Shannon entropy, where the first two terms are known to be the most important. I implement the Markov chain Monte Carlo method to sample an underlying molecular system, and then I use the kNN and kpN methods to estimate the entropy.

Finally, I present my related work on the molecular dynamics (MD) simulations of the solvation of an ion in water. Using the radial distribution function of water molecules surrounding an ion, obtained from the MD simulations, I find the effective radius of the ion. I also compare the results of the MD simulations with those of a stochastic ordinary differential equation (SODE) model to examine the validity of such an SODE approach. The work presented here is a first step toward combining statistical methods and computational analysis to tackle one of the very complex problems in mathematical modeling and computer simulations of biological molecules.

My detailed studies of a class of nonparametric entropy estimators and their application to molecular modeling demonstrate that these methods are promising. More work remains to improve the efficiency of some of these estimators, and to develop a complete theory of convergence. Further theories and more related methods are needed for better applications of these estimators in molecular modeling.

In this thesis, we present novel numerical algorithms in machine learning and data analysis. This thesis consists of two chapters. Each chapter is self-contained.

In chapter 1, we present novel methods for performing k-fold cross validation for ridge regression. Ridge regression is a very broadly used method for reduce overfitting in linear regression. And it is used in many aspects of data analysis. One very important question for ridge regression is to find the best regularization hyperparameter $\lambda$. However, this can be a time consuming procedure. Here we present an efficient yet numerical stable algorithm for computing the relative error of ridge regression across different $\lambda$'s. Besides that, we provide an novel algorithm for finding the best hyper parameter $\lambda$ without computing all the relative errors.

In chapter 2, we present algorithms for efficiently finding low rank approximation of a large sparse matrix with improved Lanczos Algorithm. Low rank approximation is one of the most important techniques in data analysis. In particular, people do principal component analysis based on that. We tackle this problem by finding the truncated SVD of the matrix using our improved version of Lanczos matrix. Our method guarantees that we only perform restart of the algorithm when necessary. Also, we prove a novel scheme for doing reorthogonalization during the Lanczos iteration. Besides that, we provide new stopping criteria for Lanczos algorithm directly based on the quality of the low rank approximation.

Faults show complex slip behaviors at different sections depending mainly on their stress and friction distributions. In the seismogenic zone, a fast earthquake happens when the frictional resistance to fault movement reduces faster than the decrease in elastic stress due to fault slip, and it releases seismic energy that causes ground shaking. Increases in depth, temperature and pressure change the frictional properties from velocity weakening to strengthening. This deeper section is referred to as the creeping zone and shows stable sliding without any stress drop. In between, there is a transition zone where slow slip can occur on asperities embedded in the creeping region [Bartlow et al, 2011; Obara et al., 2011; Ghosh et al., 2012]. The slip cannot reach high enough velocities to produce regular earthquakes, but sometimes it is still able to radiate low amplitude and low frequency seismic waves [Peng & Gomberg, 2010]. Seismicity in the seismogenic zone can trigger slow slip in the transition zone. It can also change the stress and accelerate or decelerate the seismicity on adjacent faults or even on faults hundreds to thousands kilometers away when the earthquake is large enough. Conversely, slow slip in the transition zone can also change the surrounding stress field and increase the stress in the up-dip seismogenic zone, potentially advancing the timing of earthquake failure.

In this dissertation, I study the broad spectrum of fault behaviors and explore the potential relationships between them. In Chapter 1, we give an introduction to fast and slow earthquakes and briefly introduce the main methods used to study them. In Chapter 2, I apply the back-projection method to two case studies: the 2015 Mw 8.3 Illapel earthquake, using one array with both low- and high-frequency bands imaging multiple rupture patches, and the 2015 Mw 7.8 Gorkha earthquake using multiple global arrays to image the rupture process and detect aftershocks. The back-projection results of the Gorkha earthquake imaged by different global arrays show similar rupture processes but vary in detail. One array shows continuous eastward rupture for ~60s while other arrays show a branching rupture to the northeast at ~45s. In addition, we combine multiple global arrays to improve the resolution of the back-projection method. The higher resolution also allows us to detect 2.6 times the number of aftershocks than that recorded in the global catalog.

In Chapter 3, we apply the multi beam back-projection method (MBBP) to study the slow earthquakes of the Unalaska-Akutan region in the Alaska-Aleutian subduction zone. We detect near-continuous tremor and low frequency earthquakes for nearly two years. The slow earthquakes are distributed heterogeneously in three clusters and are located deeper than those in other subduction zones. The tremors show both short and long-term migrations along strike and dip directions with a wide range of velocities. In addition, tremors and LFEs show strong spatio-temporal correlations. They are located in the same patches, and when there are LFEs bursts during tremor signals. We also observe some cases where local or regional earthquakes can terminate or amplify slow earthquake activity.

In Chapter 4, we use the move-max matched filter method to detect small local earthquakes along the San Jacinto fault (SJF) zone that are triggered by the 2014 Mw 7.2 Papanoa earthquake. Using the catalog events as templates, the move-max matched filter method detects 5.4 times the number of earthquakes recorded in the ANSS and SCSN catalog, while the matched-filter method only distinguishes 3.2 times the number of catalog events using the same detection threshold. After relocation using hypoDD, we find a new normal fault with strike almost perpendicular to the SJF. More than one mechanism may be responsible for triggering earthquakes. The transient dynamic stresses may have triggered slow slip or fault creep, and lead to the increased and protracted seismicity along the San Jacinto Fault (SJF). In addition, the time-dependent acceleration to failure process initiated by the dynamic stress change can result in the enhanced seismicity on the new fault.

The communication between the 5’ 7-methylguanosine cap structure and the 3’ poly(A) tail of eukaryotic mRNAs plays a crucial role in translation regulation. Scientists hypothesized that mRNAs interact with the cap-binding protein eIF4E, the poly(A)-binding protein PABP1, and eIF4GI through interactions with the 5’ cap structure and the 3’ poly(A) tail. These interactions bring the 5’ end and the 3’ end of the mRNA into proximity known as the eukaryotic “closed-loop” model. The formation of this “closed-loop” structure allows the physical communication between the 5’ end and the 3’ end of the mRNA, and it is thought to enhance translation initiation. Although the “closed-loop” model is a widely accepted hypothesis, there is still conflicting evidence both supporting and opposing this model.

To further elucidate this model, we tested the formation of the “pseudo-closed-loop” structure with human proteins in vitro using the electrophoretic mobility shift assay (EMSA). However, instead of forming the single “pseudo-closed-loop” structure, our results show that two non-interacting complexes are formed in vitro: the 5’cap•eIF4E•eIF4GI complex and the poly(A)-tail•PABP1•eIF4GI complex. The formation of these two separate complexes suggests that the binding of eIF4GI to eIF4E and PABP1 is competitive and mutually exclusive. Overall, we hypothesize that the formation of the “pseudo-closed-loop” structure may not occur in all mRNA transcripts or all contexts of eukaryotic translation initiation.

Motivated by biological models of solvation, this dissertation consists of analysis of models of electrostatic free energy of charged systems that incorporate both continuum and discrete idealizations of charges. Discrete models of charge can yield vastly divergent results than the corresponding continuum models for systems with a small number of particles, but will

be shown in Chapter 2 to be asymptotically equivalent in the large particle number limit.

In Section 2.1 the energy of a given continuum charge distribution is shown to be representable as a limit of approximating discrete charge distributions by way of properties of harmonic functions and Riemann sum approximations. In Section 2.2 the problem is reversed in that a given sequence of collections of point charges is shown to have a limiting continuum charge density with the same limiting electrostatic energy.

Motivated by application to a minimization problem common in molecular

modelling, potential theory, and fluid mechanics, Chapter 3 details a delicate multiscale construction to generalize the results of Chapter 2 to more general measures, requiring the further development of the theory of Radon measures and their Fourier

transforms, facilitated by a gradient flow evolution of the domain.

The analysis of Chapter 4 concerns a hybrid model of solvation that incorporates statistical mechanics and the classical Coulomb energy of a system, allowing for both continuous and discrete distributions of charge whose equilibrium configuration is described by the Poisson–Boltzmann Equation. Motivated by application to optimization, a modified free energy functional is constructed by way of a Legendre transform and is shown to be equivalent.

Despite the long history of competition between these models, a precise treatment of the question has never been addressed, to this author’s knowledge. This work is significant in bridging the gap between scales, and furthermore has application to

a wide variety of physical and biological modelling problems.