Skip to main content
eScholarship
Open Access Publications from the University of California

UC Davis

UC Davis Electronic Theses and Dissertations bannerUC Davis

Validating Generative AI for Health Communication Research: Identifying and Prioritizing Parental HPVP Vaccine Hesitancy Beliefs

Abstract

Health communication researchers are increasingly utilizing generative artificial intelligence (AI) and large language models (LLMs) in the research process, often before shared standards exist for validating outputs. Validation is especially important in formative research because the beliefs selected as message targets shape what a campaign can address. This dissertation develops and validates a reproducible, human-in-the-loop method for identifying which beliefs an audience holds and which are most strongly associated with vaccination intention. It also tests where model output can support that work and where human data remain necessary. Parental hesitancy toward adolescent human papillomavirus vaccination among rural U.S. parents serves as the test case. Study 1 uses GPT models to extract parental hesitancy beliefs from 519 published studies across 61 countries and synthesize them into a taxonomy, with human validation at each stage. Extraction was highly accurate for factual study characteristics and moderately accurate for belief statements. Human coders applied the finished 104-category taxonomy with high reliability. A small set of beliefs accounted for most documented concern worldwide, and thirteen categories appeared in all six WHO regions. Study 2 administers a 39-item instrument derived from the taxonomy to 372 rural parents and ranks beliefs by prevalence and association with vaccination intention using Hornik and Woolf’s percentage-to-gain approach. Concerns about vaccine safety, institutional trust, and perceived necessity ranked highest as candidate message targets, while practical barriers ranked lowest. Provider recommendation was associated with higher intention but did not resolve the leading concerns. Priorities also differed across subgroups defined by political ideology, child age, and race or ethnicity. Study 3 tests whether that prioritization step can be simulated. Four large language models answer the same instrument as simulated versions of the surveyed parents. Simulated parents recovered the human ranking above chance in 23 of 24 configurations, but no configuration reached the agreement expected from a second real sample. Error originated primarily in the simulated beliefs, which showed miscalibrated prevalence, compressed variance, and amplified belief-intention associations. The simulations also flattened subgroup differences that would matter for tailored messaging. Together, the studies show that large language models can accelerate belief identification when embedded in a human-validated pipeline.

Main Content

This item is under embargo until September 11, 2032.