This thesis is concerned with the topic of generalization in large, over-parameterized machine learning systems---that is, how models with many more parameters than the number of examples they are trained on can perform well on new, unseen data. Such systems have become ubiquitous in many modern technologies, achieving unprecedented success across a wide range of domains. Yet, a comprehensive understanding of exactly why modern machine learning models work so well has alluded fundamental understanding. Classical approaches to this question have focused on characterizing how models will perform in the worst-case. However, recent findings have strongly suggested that such an approach is unlikely to yield the desired insight. This thesis is concerned with furthering our understanding of this phenomenon, with an eye towards moving beyond the worst-case. The contents of this thesis are divided into six chapters.

In Chapter 1, we introduce the problem of generalization in machine learning, and briefly overview some of the approaches---both recent and classical---that have been taken to understand it.

Chapter 2 introduces a novel analyses of deep neural networks with positive homogeneous activation functions. We develop a method to provably sparsify and quantize the parameters of a model by sampling paths through the network. This directly leads to a covering of this class of neural networks, whose size grows with a measure of complexity we call the "path norm". Using standard techniques, we can derive new worst-case generalization bounds that improve on previous results appearing in the literature.

In Chapter 3, we take a critical look at the worst-case approach to understanding generalization. To do this, we develop a methodology to compute the full distribution of test errors for interpolating linear classifiers on real-world datasets, and compare this distribution to the performance of the worst-case classifier on the same tasks. We consistently find that, while truly poor, worst-case classifiers indeed exist for these tasks, they are exceedingly rare---so much so that we expect to essentially never encounter them in practice. Moreover, we observe that as models become larger, test errors undergo a concentration around a critical threshold, with almost all classifiers achieving nearly the same error rate. These results suggest that the worst-case approach to generalization is unlikely to describe practical performance of large, over-parameterized models, and that new approaches are needed.

In light of our findings in Chapter 3, in Chapter 4 we ask a complementary question: if modern machine learning systems perform nowhere near the worst-case, how close might their performance be to the best-case? For an arbitrary classification task, we can quantify the best possible error attainable by any model with the Bayes error rate, though it is generally intractable to estimate for realistic datasets. To address this intractibility, we first prove that the Bayes error rate is invariant under invertible transformation of the input features. We then use normalizing flows to estimate an invertible map between the target data distribution and a simple base distribution, for which the Bayes error can be easily computed. We then evaluate a variety of state-of-the-art architectures against this estimated Bayes error rate, and find that in some (but not all) cases, these models achieve very close to optimal error rates.

In Chapter 5, we investigate average-case generalization via ensembling---a popular method wherein the predictions of multiple models are aggregated into a single, often more powerful predictor. In classical settings, the effectiveness of ensembling is well-understood; however, for ensembles comprised of deep neural networks, the benefits of this method are surprisingly inconsistent. To understand why, we first prove a new set of results relating the ensemble improvement rate (a measure of how much ensembling decreases error relative to the average error rate of a single model) to the ratio of model disagreement to the average error rate. This results in new oracle bounds on the error rate of ensemble classifiers, significantly improving on prior results in the literature. We then investigate ensembling experimentally and find, most notably, a distinct and consistent transition in the rate of ensemble improvement (and the disagreement-error ratio) occuring at the interpolation threshold---the point at which individual classifiers achieve exactly zero training error. Our findings suggest that ensembling is significantly less effective in the modern, over-parameterized regime than it is in more classical settings.

Finally, in Chapter 6 we conclude with some reflections on the state of the field, and outlook for how it might advance in the coming years.

Many processes that occur at biological membranes have been shown to be remarkable displays of molecular coordination and shape transformation. Despite the apparent complexity of these processes, investigations of simplified systems have successfully characterized many ways that these materials can self-organize. Such studies established that proteins may either bend membranes by binding to them along curved structural domains or by inserting into them asymmetrically. One recent study, however, showed that generic clusters of small membrane-adhered proteins that lack these features can nevertheless remodel membranes into highly curved shapes.

In this manuscript we use theoretical and computational methods of statistical mechanics to elucidate how a driving force for membrane curvature emerges from the steric effects among such adhered proteins. First, we develop a basic thermodynamic model of the experimental system referenced above, considering the energetic penalty of membrane bending for simple shapes while also introducing the 2-dimensional pressure of the adhered protein layer. We show that increasing the curvature of the underlying membrane effectively increases the area of the adhered protein layer, lowering the associated pressure and entropically stabilizing the composite system at nonzero curvatures. To generate quantitative predictions of curvature as a function of adhered protein concentration, we model the protein layer as a hard disk fluid and compare these results to experimental data. We also use overdamped Langevin dynamics simulations, again considering only volume excluding interactions among proteins, to observe shape transformations and fluctuations of such systems in more complex morphologies. We extend our thermodynamic model to show that cylindrical curvature becomes more stable than spherical curvature at protein densities higher than some critical value. This critical value becomes smaller as membrane area increases, which supports experimental observations and explains why tubules are unstable in simulated systems, which are much smaller.

The development of a molecular simulation model for this research was a significant effort, and we discuss this implementation in some detail. We build upon a coarse-grained model previously developed in our research group, however we modify its original functional forms to be smoothly varying and thus stable for dynamical integrators. We also develop consistent coarse-grained models for adhered proteins and for a second lipid type. We then discuss extensions developed for future studies, methods of analysis, and relaxation time scaling for cylindrical configurations.

Finally, we show how the adhered protein layer effectively renormalizes the original material properties of the underlying membrane. We begin by theoretically analyzing our composite membrane-protein system in the regime of small but nonzero curvature fluctuations. This analysis yields a form identical to the standard free energy of a bare membrane but with effective expressions for the bending rigidity, surface tension, and spontaneous curvature that are dependent on the density of adhered proteins. As protein density increases, the effective bending rigidity and spontaneous curvature reduces until the system can become unstable. We support these predictions with molecular simulation calculations of the type discussed earlier.

These results exemplify the ideas discussed at the start of this manuscript, demonstrating how the emergent characteristics of seemingly complex membranes can sometimes be remarkably folded back into the basic physics of these soft materials. Moreover, since a membrane attachment region alone is sufficient for protein clusters to produce the results presented throughout this work, these results could characterize a new and general way that proteins modify and sculpt membranes during various essential cellular processes.

SUMMARY: Burglary location, poverty, education, income, and population density data were collected from the Los Angeles Police Department, the U.S. Census and U.S. Internal Revenue Service. Prior burglary point data and socio-demographic spatial covariates were used to construct annual kernel-intensity and Poisson point process hybrid models to predict the burglary rates of the following year. To test the utility of the spatial covariates over kernel-intensity only methods, two models were constructed: A baseline model using only kernel-intensity data, and an expanded model using kernel-intensity data and additional spatial covariates. Analysis-of deviance test revealed a significant difference of 116.64 with nine degrees-of-freedom (p-value < 0.001). Voronoi residuals used for cross-validation indicated a 2.608% improvement in the area adjusted-root-mean-square error.

Despite the high efficiency and long development time of blue-emitting gallium nitride (GaN)-based μLEDs using metal-organic chemical vapor deposition (MOCVD) growth methods, high-performance devices emitting at the longer visible wavelengths have seen comparatively far less research and investment. Difficulties surrounding the growth of high-In layers by MOCVD makes device improvement increasingly difficult as the wavelength increases, requiring higher In content active regions, causing structural degradation and poor electrical performance. In this work, first higher power and efficiency of green-emitting LEDs is achieved through MOCVD design along with advanced characterization and device physics. Experimental results, modelling, and advanced design are employed in tandem to achieve higher power and lower operating forward voltage. The knowledge generated and advances made in the green LED design are then leveraged into realization of red-emitting InGaN/GaN μLEDs. Few examples of InGaN/GaN devices emitting in the red spectral regime currently exist with only a few realized proof of concept devices. Here, by employing semi-relaxed InGaN substrates, true red μLEDs using an InGaN growth structure can be achieved. Initially, a method to significantly improve material quality from the starting substrate is established, fully eliminating large-scale extended defects known as V-defects initially present in the substrate. On this newly improved material, red-emitting μLED devices with appreciable external quantum efficiency (EQE) can then be realized. Growth conditions in the active region and p-type regions are both thoroughly explored in order to increase light output power of the device and in turn improve EQE. These developments then finally result in realization of a red-emitting device nearing 1% peak EQE.