UC Irvine

Modern machine learning systems increasingly operate across multiple stages of the lifecycle: large-scale training over distributed data sources, continual adaptation as data distributions evolve after deployment, and inference-time decision making for foundation model applications under latency and cost constraints. Across these stages, systems must operate under practical constraints, including data-sharing regulations, data heterogeneity, non-stationary environments, latency budgets, and limited computational resources. This dissertation offers a unified view of the machine learning lifecycle through the lens of these operational decisions, and argues for a dedicated middleware layer: a set of principled decision policies that governs behavior across training, continual adaptation, and inference. In this view, middleware is not merely a communication conduit or infrastructure plumbing layer, but the decision brain between machine learning applications, models, and their operating environments. It interprets available system and model signals, reasons over constraints, and coordinates adaptive behavior. First, to achieve robust training across data and platform heterogeneity, we present techniques for federated learning that leverage participant-level data characteristics to improve coordination across distributed clients. By analyzing label distribution structure, training can better preserve diversity, improve convergence, and reduce communication overhead. Second, for continual learning under distribution shifts, we develop mechanisms that monitor evolving data conditions and selectively specialize models as environments change. By using learned representations and label statistics to detect shifts, deployed systems can adapt without the expense of full retraining. Third, for inference-time optimization, we study dynamic context management for large language and vision-language models. We show how outputs such as log probabilities, confidence estimates, and lightweight metadata can guide decisions, including example ordering, retrieval fidelity selection, and cost-aware context construction. Although these settings appear distinct, they share a common systems pattern: machine learning systems already generate rich internal signals that reflect model state, data state, and execution conditions, yet these signals are rarely exposed as first-class abstractions for system control. This dissertation argues that treating such signals as actionable primitives enables a new decision layer for machine learning systems — one that can observe, analyze, and act on model-generated evidence to build adaptive policies across training, deployment, and inference, thereby enabling efficient, robust, and self-adaptive machine learning systems.