This study explores the implications of different modeling choices when predicting mortalityduring intensive care visits using recurrent neural networks. Using the MIMIC-III database, models were trained and tested with varying memory cells, architectures, and other hyper- parameters. Performance gains from incorporating information from unstructured clinical notes was tested as well. The study finds that a range of relatively shallow networks with varying memory cells and architectures can perform well and produce similar results, all of which outperform traditional mortality risk scores such as SAPS II. Adding information from clinical notes boosts model performance even with a simple natural language processing algorithm. Although methodological differences make direct comparisons complicated, the most accurate model presented here achieves an AUROC score of 0.943 which represents a slight improvement over similar prior work.

This thesis explores the role of inductive biases in multi-stage machine learning problems. Modern machine learning often involves multiple steps of preprocessing, training, and adaptation and models may be deployed to make many decisions over time. These complex pipelines can obscure the impact of specific biases in the final model's performance. In chapter 2, we investigate the role of batch active learning in graph-based semi-supervised learning. Through theoretical motivation and empirical validation, we demonstrate improved accuracy and efficiency. In chapters 3 and 4, we investigate the role of stratification in non-negative matrix factorization and tensor factorization. We develop efficient multiplicative-update algorithms and demonstrate their effectiveness on synthetic and real-world datasets. In chapter 5, we investigate the role of topological message-passing in relational structures. We propose a unifying framework for topological message-passing networks and demonstrate its effectiveness in mitigating oversquashing. This framework unifies many topological deep learning (TDL) methods under a common axiomatic framework, allowing for consistent theoretical analysis and greater understanding of the algebraic and topological tools employed in TDL. In chapter 6, we investigate the role of zero-shot context generalization in reinforcement learning. We propose a novel method for zero-shot context generalization and demonstrate its effectiveness in improving model performance. This provides a straight-forward extension of many off-policy reinforcement learning methods, which improves generalization to unseen contexts. Through these investigations, we provide a comprehensive theoretical and empirical analysis of the aforementioned inductive biases in multi-stage machine learning problems. Our findings highlight the critical role of these biases in enhancing model performance and their broad applicability across diverse domains.