UC Berkeley

Modern deep learning has succeeded in regimes where data is abundant and high-quality, the training and deployment distributions match, and the analyst keeps close control of the modeling pipeline. Real-world deployments rarely satisfy all of these conditions. This dissertation studies three such hard regimes for deep learning, distribution shift, agentic data science, and data scarcity, through the lens of the Predictability--Computability--Stability (PCS) framework for veridical data science. The first project addresses distribution shift in the simplest tractable setting. We prove the first non-asymptotic excess risk bounds for benignly-overfit minimum-norm linear interpolators evaluated on a target distribution that differs from the source. From these bounds we propose a taxonomy of beneficial and malignant covariate shifts parameterized by the degree of overparameterization, and corroborate the taxonomy empirically on real image data and on fully-connected neural networks. The second project tackles agentic data science, where a large language model executes the entire analytical workflow and the analyst can no longer audit the intermediate decisions. We propose a pair of lightweight sanity checks that apply targeted perturbations to the data and the analytic prompt and ask whether the agent's conclusion survives. The checks are validated on synthetic data with controlled signal-to-noise ratios and applied to eleven real-world datasets, identifying six on which an affirmative agentic conclusion is not well-supported. The third project develops a deep learning pipeline for pediatric focused assessment with sonography for trauma (FAST), a bedside ultrasound exam used to detect hemorrhage from intra-abdominal injury (H-IAI) in injured children. The pipeline first localizes Morison's pouch and then classifies frames for hemorrhage. We evaluate on a pediatric cohort of 207 exams, considerably larger than previous pediatric FAST datasets but containing only 17 H-IAI positive cases. We use Monte Carlo cross-validation to report metrics stable under this small positive count.