Generative and Composable User Interfaces via Task Structures
Skip to main content
eScholarship
Open Access Publications from the University of California

UC San Diego

UC San Diego Electronic Theses and Dissertations bannerUC San Diego

Generative and Composable User Interfaces via Task Structures

Abstract

Human information work often unfolds through iterative, exploratory, and partially specified processes. Yet current digital workspaces do not reflect the evolving structures of the task itself; instead, they are organized around fixed tools, specific content types, and application boundaries. This mismatch fragments work across applications and leaves coordination burdens to the users. This design paradigm has persisted partly because the fluid, idiosyncratic nature of tasks made them difficult to represent directly. As AI systems become increasingly capable of interpreting and externalizing the structure of ongoing work, this constraint begins to loosen, opening the possibility of interfaces that are not static containers for content, but dynamic representations of the task itself. This thesis proposes task structures as an underlying representation for a new generation of generative and composable interfaces. Task structures make explicit the objects and relationships that constitute ongoing work. By transforming these structures into representations that are directly manipulable by users and accessible to AI systems, interfaces can move beyond static layouts to become shared representations that support more continuous and context-aware collaborative information work. My PhD work is developed in two parts. The first investigates how task structures can be identified and leveraged to design interfaces for specific activities: DataParticles supports animated unit visualization authoring with a single anchoring structure; VideOrigami supports video creation through a workspace that coordinates multiple structures. The second part explores how this approach generalizes to broader, open-ended tasks. Jelly introduces a generative interface pipeline that constructs object--relational task structures from user intent and maps them into malleable, reusable interfaces. Together, these systems demonstrate how interfaces can be derived from and evolve with the structure of work itself. This thesis takes a step toward a shift from tool-centered interfaces toward task-driven systems that evolve alongside user activity, enabling more fluid information work and situated forms of human-AI collaboration.