UC Berkeley

Artificial intelligence (AI) is increasingly used by individuals, firms, and organizations around the world, offering an opportunity to expand capabilities and improve livelihoods. Yet adopting and adapting to any new technology is difficult, and these challenges are compounded when the technology is as complex as AI. This dissertation examines how AI could shape strategic decision processes, with a focus on understanding the implications of AI system capabilities for low-resource contexts. Underlying the dissertation is the idea that if we want AI systems to have a positive impact, we must begin by understanding model performance. My first two essays explore this within the context of AI-generated advice: if we can understand where generative AI provides high-quality advice, then we can identify cases where AI should be deployed and cases where additional input may be needed in order to be beneficial to the user. I start by exploring this within the context of a field study in Kenya which varies the extent to which individuals receive advice from an AI system as opposed to traditional human advice. Results from this study suggest that AI-generated advice and expertise may reinforce each other, while highlighting the risks that can emerge if individuals begin to use AI at the expense of vetting the advice they receive by engaging with external sources of information. I then explore the extent to which the recommendations produced by AI systems may inadvertently privilege some contexts over others. Specifically, I test whether the default response from AI systems provides higher quality advice to individuals living in high- versus low-resource contexts. Results from this study provide evidence that default responses from AI are more aligned with high-resource contexts, which I document both within and across countries. However, I find that much of the difference in the quality of responses provided by AI stems not from fundamental differences in the model's underlying capabilities, but instead from the default context assumed by the model. Finally, I examine the performance of AI in the context of a specific use case, evaluation. I develop a test of whether AI systems prefer proposals that were written by AI through a novel design which holds proposal quality constant, varying only the extent to which language inordinately used by AI models is included in the proposal. I provide evidence of self-preference in model evaluation, and show that these distortions are substantial enough to change the ranking of proposals, while also highlighting that because of the low cost of using AI for evaluation, it can still be beneficial to use AI systems, even given these errors, if the alternative evaluation strategies are sufficiently costly. Together, these essays highlight the potential of generative AI systems to improve people's livelihoods, while also cautioning that understanding the capabilities of these systems is fundamental to using them effectively. By better understanding AI system capabilities, we can identify when more contextual information or guidance is needed, and when AI advice, while imperfect, may still be efficient due to its lower cost. As the capabilities of AI systems continue to rapidly improve, ensuring that these tools are deployed with the right enabling resources will be an important part of realizing their potential for people around the world.