Skip to main content
eScholarship
Open Access Publications from the University of California

UCLA

UCLA Electronic Theses and Dissertations bannerUCLA

Closed-Loop Multimodal Generative Artificial Intelligence under Real-World Constraints

Abstract

Multimodal generative models are increasingly used in interactive and scientific settings where success requires more than perceptual sample quality. A deployable system must evaluate structured behavior, align model optimization with graded objectives, and generate reliable outputs when training data or measurements are imperfect. This dissertation develops these three capabilities as a closed loop of evaluation, alignment, and generation.The first part studies evaluation for instruction-based image editing. EdiVal-Agent decomposes scenes into objects and evaluates instruction following, content consistency, and visual quality across multiple editing turns. MT-EditFlow then uses these multi-axis signals to optimize sequential editing trajectories and reduce exposure bias and error propagation.The second part studies alignment. REAL formulates regression-aware reinforcement learning for language models used as numeric judges, preserving the ordinal information discarded by binary rewards. EBAMA instead aligns text-to-image diffusion through object-conditioned energy-based attention-map objectives that improve attribute binding and reduce object neglect. The third part studies generation under corrupted information. Restoration Score Distillation learns one-step generators using only degraded observations, while Flow Priors uses flow-matching likelihood structure to solve linear inverse problems through iterative corrupted trajectory matching. Together, these methods show how evaluation signals, alignment objectives, and generative priors can be designed around real-world constraints rather than idealized clean supervision.