Audience participation has appeared prominently as a critical response to spectatorship in various artistic fields since the second half of the twentieth century. In music, a variety of participatory approaches have been developed that provide the audience with varying degrees ofcreative agency. Each approach fosters certain relationships and communications among the participants, giving rise to a range of social dynamics and politics. The objective of this dissertation was, first, to develop a new approach that involved audience members interacting with a limited number of objects placed in a concert space to affect sonic characteristics and theperformance of an ensemble that consisted of featured musicians, and second, to analyze the social dynamics and politics such an approach gave rise to.

Literature and creative works in the fields of music, visual arts, and theater were used todevelop a method of viewing and assessing the politics of participation that was used to critique the creative portion of this dissertation that took the form of two concerts of interactive musictitled Space Within. The production of the concerts involved development of hardware and software to enable audience interaction and mediation between the participants and theensemble. Scenic, lighting and projection design also played important roles in facilitating these aspects.

Several members of the audience were interviewed following the concerts while others were asked to respond to a survey to understand how they perceived their roles in the concerts and their impression of the social dynamics the event fostered. While most of them felt that their presence was consequential in the sonic outcome, not all of them were certain how exactly theyaffected the outcome. I conclude that while the audience may not have “co-authored” the sounds heard during the event, the event was successful in drawing their attention to and persuading them to reconsider their roles in a musical context.

Spatially augmented reality (SAR) systems illuminate objects or environments of different shapes or size, starting from table top objects to room-sized environments, using multiple projectors that are casually aligned creating full or partial overlaps with their neighbors. Full overlaps or superposition is used to increase brightness and partial overlaps or tiling is used to increase resolution and scale. Such systems enable applications such as visualization, entertainment and education by projecting data right on the physical 3D model.

Creating a seamless display on complex shapes using multiple projectors entail two primary challenges. Geometric registration which addresses pixel alignment in the overlap of multiple projectors and warping to conform to underlying 3D shape and color registration addresses the spatial variation of color within and across different projectors and also color variation introduced by the complex non-planar shape of the 3D surface.

My thesis presents un uniform approach for consistent calibration of SAR systems enabling easy deployment of spatially augmented reality environments of different sizes (from small objects to large immersive environments), different shapes (inside-looking-out or outside-looking in), and different configurations (tiled or superimposed). First We show that a unified approach that relies on cross-correlation and validation between the devices (projectors and cameras) and display properties achieves a consistent system reconstruction resulting in a highly accurate, robust and scalable registration in multi-projector systems that is not possible by independent calibration of devices and reconstruction of surface properties. Second we present the first method to achieve complete color seamlessness (both luminance and chrominance) for multi-projector displays on completely arbitrary shapes using a novel shape-aware 3D color gamut morphing method. Finally we present multiple interaction modalities, namely interaction of the handheld projector (a) with the surface geometry only, (b) with overlaid digital content only (from other static or dynamic handheld projectors), and (c) with both the digital content and the surface simultaneously.

Artificial Intelligence (AI) systems that use language generation models hold incredible promise to assist humans to perform complex decision-making tasks. State-of-the-art language generation models can produce engaging content, reason about the world, and retrieve relevant information for an information-seeking task. However, these models often ignore sparse, long-tail knowledge about individual users, cultural subtleties, and domain-specific knowledge, preventing end users from reaping the full benefit of the scale. In this dissertation, we redesign AI systems to start with individual needs.

Ideally, a user-centric AI system must be grounded in the real-world, produce faithful chains of reasoning to explain its prediction, and align with the user's preferences. We elevate existing AI systems with knowledge, explanations, and interactions and develop both training-time and post-hoc techniques to make these systems user-centric.We show additional knowledge-grounding promotes user success in achieving conversational goals while using a conversational AI system. We demonstrate that AI explanations, when attributed to world knowledge, render them to be faithful and consistent. Finally, we discover that user-centric interventionist approach can help users obtain more equitable predictions backed by faithful explanations as compared to a black-box counterpart. In summary, our research establishes that increased effectiveness, explainability, and equitability can be achieved through knowledge-grounding and user-centric approaches to personalize AI models---a long-standing goal of artificial general intelligence.

Audience participation has appeared prominently as a critical response to spectatorship in various artistic fields since the second half of the twentieth century. In music, a variety of participatory approaches have been developed that provide the audience with varying degrees of creative agency. Each approach fosters certain relationships and communications among the participants, giving rise to a range of social dynamics and politics. The objective of this dissertation was, first, to develop a new approach that involved audience members interacting with a limited number of objects placed in a concert space to affect sonic characteristics and the performance of an ensemble that consisted of featured musicians, and second, to analyze the social dynamics and politics such an approach gave rise to.

Literature and creative works in the fields of music, visual arts, and theater were used to develop a method of viewing and assessing the politics of participation that was used to critique the creative portion of this dissertation that took the form of two concerts of interactive music titled Space Within. The production of the concerts involved development of hardware and software to enable audience interaction and mediation between the participants and the ensemble. Scenic, lighting and projection design also played important roles in facilitating these aspects.

Several members of the audience were interviewed following the concerts while others were asked to respond to a survey to understand how they perceived their roles in the concerts and their impression of the social dynamics the event fostered. While most of them felt that their presence was consequential in the sonic outcome, not all of them were certain how exactly they affected the outcome. I conclude that while the audience may not have “co-authored” the sounds heard during the event, the event was successful in drawing their attention to and persuading them to reconsider their roles in a musical context.

This dissertation addresses a growing challenge of visualizing and modifying massive 3D geometric models in a collaborative workspace by presenting a new scalable data partitioning algorithm in conjunction with a robust system architecture. The goal is to motivate the idea that utilizing a distributed architecture may solve many performance related challenges in visualization of large 3D data. Drawing data from modeling, simulation, interaction and data fusion to deliver a starting point for scientific discovery, we present a collaborative visual analytics framework providing the abilities to render, display and interact with data at a massive scale on high resolution collaborative display environments. This framework allows users to connect to data when it is needed, where it is needed, and in a format suitable for productivity while providing a means to interactively define a workspace that suits one's need. The presented framework uses a distributed architecture to display content on tiled display walls of arbitrary shape, size, and resolution. These techniques manage the data storage, the communication, and the interaction between many processing nodes that make up the display wall. This hides the complexity from the user while offering an intuitive mean to interact with the system. Multi-modal methods are presented that enables the user to interact with the system in a natural way from hand gesture to laser pointer. The combination of this scalable display method with the natural interaction modality provides a robust foundation to facilitate a multitude of visualization and interaction applications.

The final output from the system is an image on a large display made up of either projection or lcd based displays. Such a system will have many different components working together in parallel to produce an output. By incorporating computer graphics theory with classical parallel processing techniques, performance limitations typically associated with the display of large or numerous items on multiple display devices and multiple input sources are overcome.