UC Berkeley

Despite their increasingly widespread usage, large language models (LLMs) do not meet all needs of all users in practice. How can we bridge this gap to design LLMs that work reliably and fairly for the broad range of real LLM users? I present research tackling this problem at three stages of model design. First, I discuss how to build LLMs that leverage disagreement among user preferences to work for entire populations of users, using that disagreement as signal to improve model training and evaluation. Then, I discuss designs for rigorous evaluations to extricate challenging harms that diverse users face when using LLMs. Finally, I discuss addressing core technical failures of LLMs, such as miscalibrated confidence, to reduce downstream risks when models are deployed to users with different needs. Combined, these interventions facilitate building LLMs that minimize societal harms, and maximize benefits to a wider range of real-world users.