UC Santa Barbara

Large language models (LLMs) such as GPT-4, DeepSeek-V3, and Gemini have demonstrated remarkable capabilities in language understanding and generation. However, accurately evaluating their performance remains a significant challenge, as traditional metrics like BLEU and even crowd source human assessments often prove unreliable. This underscores the need for robust evaluation frameworks tailored to the unique complexities of LLMs. Beyond evaluation, refining LLM capabilities through post-training techniques—such as reinforcement learning from human feedback (RLHF), inference-time optimization, and knowledge distillation—is crucial. The effectiveness of these methods hinges on the quality of evaluation frameworks and algorithmic design, as flawed feedback or poor algorithms can amplify biases or misalign models with human values. This thesis tackles these challenges by focusing on two key areas: evaluation and efficient post-training. It investigates five research questions: (1) Can LLM fairly evaluate other LLMs and themselves? 2) Can we learn a reliable and explainable metric to evaluate LLM and text generation without human annotated data? 3) Can we effectively improve LLM generation at inference time? 4) Can we better and more efficiently align LLMs with human feedback? 5) Can we better distill large LLMs into small ones? Through novel metrics and post-training techniques, this work advances the science of LLM evaluation and post-training, offering strong empirical evidence to improve the performance of state-of-the-art LLMs.