UCSF

The burgeoning interest in leveraging deep learning within the medical field heralds a promising frontier for enhancing disease understanding and patient care. Yet, this technological advance is not without its challenges. One significant issue is the underutilization of diverse data types; medical records and biological factors, while crucial, do not encompass the entirety of necessary information. Social Determinants of Health (SDoH), for instance, play a pivotal role in disease comprehension but are often neglected in research. Furthermore, while deep learning holds potential for diagnosis and aiding clinical decisions, the absence of rigorous external validation undermines its reliability. Many models, despite performing well in initial settings, falter under broader, real-world scrutiny. Additionally, the tendency to harness large datasets and maximize feature inclusion for disease analysis sometimes overshadows the value of engineered features. These more targeted, hypothesis-driven attributes can sometimes offer clearer insights into disease mechanisms, a nuance that is frequently overlooked in the rush towards big data approaches.

These challenges manifest distinctly across different data modalities in medical research. In the realm of Electronic Health Records (EHR), the exploration of disease mechanisms often prioritizes medical data, inadvertently sidelining non-medical but equally vital Social Determinants of Health (SDoH) such as financial stability, mental health, and physical activity. This oversight can skew our understanding of disease etiology and patient outcomes. In medical imaging, the rapid development and deployment of deep learning models boast of enhanced diagnostic accuracy. Yet, this domain is particularly susceptible to the pitfalls of insufficient external validation. Minor perturbations or "noise" within the imaging data can dramatically compromise the predictive reliability of these models, emphasizing the need for robust validation processes. Genomic studies, on the other hand, face the challenge of signal dilution amidst the vast array of genomic features. The pursuit of correlations across tens of thousands of genes often overlooks the critical influence of covariates and noise, potentially obscuring the true biological signals vital for understanding disease processes. Each of these issues highlights the complexity of medical data analysis and the need for nuanced approaches that consider the full spectrum of relevant factors.

This dissertation is dedicated to the development and application of innovative computational strategies, employing practical deep learning techniques to address these prevailing challenges. Firstly, it underscores the necessity of integrating comprehensive and meaningful features in deep learning research, with a particular emphasis on the inclusion of Social Determinants of Health (SDoH) factors, to present a more holistic view of disease mechanisms. Secondly, it demonstrates the imperative role of high-quality data, coupled with human feedback and rigorous external validation, in enhancing the reliability and applicability of deep learning frameworks within the medical domain. Thirdly, the dissertation advocates for the strategic use of high-level feature engineering, as opposed to relying on an overwhelming volume of features, to decipher complex biological systems.