UC San Diego

From Paleolithic etchings to computational graphics, visualization technologies that enable us to externalize our thoughts have been critical to the communication of ideas to others and across generations. Not only have renderings like simple line-drawings been prolific across cultures, but in recent years, generative text-to-visual systems have dramatically decreased production barriers and have increasingly populated our modern world with rich artifacts of visual communication. The study of visual communication—how people express their knowledge in visual form—presents abundant opportunities and challenges to explore core mechanisms of human communication systems, because it relies on complex interactions between perception and pragmatics. What guides what visualization strategies people use to convey their ideas to others through visual form? My dissertation introduces new experimental methods to explore these strategies. Overall, I find that people’s visual production behaviors shift what kind of information they prioritize depending on their task goals and find that their representational choices directly impact downstream interpretation by viewers. In Chapter 1, I investigate how communicative goals and immediate sensory inputs jointly determine the kind of visual information that people include in their drawings and find that people flexibly adapt their behavior to these different task goals by prioritizing different semantic information about visual object concepts. In Chapter 2, I explore how people adapt their visualization strategies when producing diagrams (called “visual explanations”) of higher-level knowledge (e.g., object function vs. object appearance) and find that while these strategies facilitate inferences about physical mechanism, they do so at the expense visual fidelity and recognition of drawn objects. To evaluate whether such flexible visual communication behavior extends to more domain-specific knowledge, Chapter 3 explores how people evaluate what makes data visualization effective for different task goals and find that people selectively prioritize different kinds of data visualization designs depending on what supports fast and accurate graph comprehension by viewers. Taken together, insights from these three lines of work help contribute towards developing a more unified theory of visual communication and, ultimately, aim to better inform the development of human-centered visualization technologies guided by cognitive theories of visual communication.