The way we listen to music has been changing fundamentally in past two decades with the increasing availability of digital recordings and portability of music players. Up to date research in music recommendation attracted millions of users to online, music streaming services, containing tens of millions of tracks (e.g. Spotify, Pandora). The main focus of up to date research in recommender systems has been algorithmic accuracy and optimization of ranking metrics. However, recent work has highlighted the importance of other aspects of the recommendation process, including explanation, transparency, control and user experience in general. Building on these aspects, this dissertation explores user interaction, control and visual explanation of music related mood metadata during recommendation process. It introduces a hybrid recommender system that suggests music artists by combining mood-based and audio content filtering in a novel interactive interface. The main vehicle for exploration and discovery in music collection is a novel visualization that maps moods and artists in the same, latent space, built upon reduced dimensions of high-dimensional artist-mood associations. It is not known what the reduced dimensions represent and this work uses hierarchical mood model to explain the constructed space. Results of two user studies, with over 200 participants each, show that visualization and interaction in a latent space improves acceptance and understanding of both metadata and item recommendations. However, too much of either can result in cognitive overload and a negative impact on user experience. The proposed visual mood space and interactive features, along with the aforementioned findings, aim to inform design of future interactive recommendation systems.

This dissertation explores the ways in which media artworks depict agency and convey life-like qualities. This exploration is centered around the design and implementation of two interactive media artworks, HIVE (2016--2018) and Cacophonic Choir (2019--2020), and makes use of a novel conceptual framework grounded in an approach to media arts practice that uses the notion of agency as a lens for examining and creating artworks. Starting with an inquiry into novel agent-based art practices that neither feature robotic nor virtual agents, the dissertation reevaluates the notion of agency in artistic contexts in light of the relatively recent establishment of sonic interaction design as a field in its own right and the renewed emphasis on materiality as the result of rapidly evolving design and fabrication technologies. To this end, I introduce Sonically Actuated Spatial (Sono-spatial) Agent practice as an area of artistic practice that fuses digitally designed and fabricated artifacts, sonic expressions, and interactive behaviors in order to create ‘perceived’ life-like systems. In summary, this dissertation aims to expand upon the theoretical and conceptual underpinnings of interactive media arts via the products of my artistic practice, as well as theoretical discussions, design methods, principles, and strategies, all of which are distilled from this practice.

This dissertation presents Stride, a language for sound synthesis, processing, and interaction design. With a novel and unique approach for handling sampling rates as well as clocking and computation domains, Stride prompts the generation of highly optimized target code. Optimization is achieved by giving the user of Stride control over the Stride code generator through its syntax. The optimizations render Stride an ideal language to target resource-constrained devices such as microcontrollers. Stride is a declarative language and adopts features from dataflow languages. With only two syntactic constructs, Stride is easy to learn. Through resource abstraction and separation of semantics from implementation, a wide range of computation devices could be targeted such as microcontrollers, system-on-chips, general-purpose computers, and heterogeneous systems. Users of Stride can write code once and deploy on any supported hardware.

After presenting the challenges of targeting resource-constrained microcontrollers with popular music programming languages in use today for sound synthesis and processing, a new programming language and its syntax are introduced to address these challenges. This is followed by demonstrating how the language enables its user to control the code generation process to yield efficient and optimized target code. Next, the semantics of the language and some of its core building blocks are presented in detail followed by the user-controlled concurrency model built into the language. Designing interaction using some of the core blocks is then presented through a set of examples followed by some of the advanced building blocks of the language. Finally, the language is presented as part of an encompassing development environment and all of its components including the integrated development environment and the compiler.

The use of space in music is a complex issue which involves several different, yet interrelated factors. The technical means of performance, the sonic material, and the overall musical aesthetic should all work together to produce a truly immersive experience and one which the listener can comprehend the spatial impression as independent and yet musically significant. Performances of spatial music typically involve a distributed audience and often take place in an acoustically reverberant space. This situation is quite different from the case of a single listener at home, or the composer in the studio or in a virtual reality setting with head mounted displays. As a result, spatial strategies which are effective in one context may not be transferable to another context. This thesis attempts to study the psychoacoustic factors of various music spatialization strategies in different contexts and in that process attempts to answer the questions, what psychoacoustic factors affect listening to music in non-virtual listening contexts and how could they transform when listening in a virtual listening context?How do these psychoacoustic factors work? This question is answered through three psychoacoustic studies. The first one on Sound Source Distance, the second and third are comparative studies of music spatialization algorithms with different music genres – a loudspeaker study and a binaural analysis study. The last part of the thesis is to build a prototype of a virtual concert hall that uses some of the learnings of the aforementioned studies.

In 1958, Iannis Xenakis used a glissandi diagram as the parti for the architectural design of the Philips Pavilion. It was a pioneering example of an architectural form directly derived from the morphology of a sound object. In stark contrast, sound is usually seen as a pollutant, as something to be absorbed, blocked and diffused in current architectural design and practice. From an architectural design theory perspective, Digital Morphogenesis demonstrates significant theoretical implications in computational design and digital fabrication due to its natural-inspired sustainable methodologies, new forms, fabrication techniques, and multidisciplinary approaches. However, our current architectural and design field does not consider sound material from a morphological perspective in both theory and practice.

My research aims to expand the theory of digital morphogenesis by encompassing sound objects as an additional architectural material for computational design and fabrication. I introduce Sound Morphogenesis Design (SMD), a new design framework that uses information theory as a foundation to develop methodologies for architecture and design research. My research explores three main questions: (1) How does architectural practice change when sound is not viewed as pollution but as a design opportunity? (2) How does sound morphology expand the theory of digital morphogenesis and change the approach for architectural design research? And (3) what would be the novel design practice of sound morphogenesis design that enhances aural experiences and aesthetics? To integrate sound morphology in design, this dissertation presents a systematic survey of musical aesthetics in architectural projects and develops two SMD methodologies: WYHIWYW and SoftTectonics. WYHIWYW (What You Hear Is What You Wear) investigates digital domain design techniques such as Digital Design through the Open-Sound-Control Protocol, Computational Design through MIDI, and Computational Design through Modular Synthesizers. Through such techniques, the listening experience enriches and expands the design process. SoftTectonics explores sound morphogenesis from three different approaches using physical prototyping of soft materials: tip-grow, feather-flip, and pneumatic-cavity. These methods provide a framework for digital architecture, wearable design, and new media artworks.

A novel analysis tool CIMT (Creative-Impact-Musicality-Technology) evaluates the developed research experiments to prove SMD’s contribution as a design theory. The present dissertation paves a path to cross music and architecture for future discovery, design, and human perception.

Many forms of life in the natural world have the extraordinary capacity to sense their environments, to learn, and to remember, just as humans do, even though they are vastly different from us. In this dissertation, I presented novel techniques developed to exhibit an interactive abstract virtual reality experience that invites viewers to see the natural world from a different perspective. I developed the vertex displacement and color turbulence approaches to showcase organisms. The organisms can also modulate their shapes according to the volumes and frequencies of sound. Furthermore, the experience displays turbulent flow on the organisms’ surface to demonstrate the concept of energy flow, or vitality, among all organisms in the natural world. Another novel feature is that viewers can interact with the surface colors through ray casting from a handheld controller.

Biometric technology has brought enhancements to identification and access control. As more digital applications request people to input their biometric data as a more convenient and secure method of identification, the possibility of losing their personal data and identities may increase. The phenomenon of biometric data abuse causes one to question what their true identity may be and what methods can be used to define identity and hidden narratives. The questions of identification and the insecurity of biometric data have become my inspiration, providing artistic approaches to the manipulation of biometric data and having the potential to suggest new directions for solving the problems. To do so, in-depth investigation of the narratives beyond the visual features of the biometric data is necessary. This content can create a close link between an artwork and its audience by causing the latter to become deeply engaged with the artwork through their own stories.

This dissertation examines narratives and artistic explorations discovered from one form of biometric data, fingerprints, drawing on insights from various fields such as genetics, hand analysis, and biology. It also presents contributions on new ways of creating interactive media artworks using fingerprint data based on visual feature analysis of the data and multimodal interaction to explore their sonic signatures. Therefore, the artwork enriches interactive media art by incorporating personalization into the artistic experience, and creates unique personalized experience for each audience member. This thesis documents developments and productions of a series of artworks, Digiti Sonus, by focusing on its conceptual approaches, design, techniques, challenges and future directions.

The present investigation aims to identify best practices to design virtual reality (VR) story-worlds to promote prejudice reduction and foster prosocial attitudes toward under-represented groups. Past research shows that VR, which allows the simulation of scenarios that would otherwise be prohibitively difficult or impossible in real life, can be an effective tool for understanding human cognition and researching perception, decision-making, and behavior (Groom, Bailenson, Nass, 2009). In tightly controlled immersive digital simulations, one can inhabit an avatar’s digital body with salient features (different skin color, age, gender, social status) to trigger beliefs, affect, and behaviors different from normal day-to-day life (Bailenson &Yee, 2007). VR perspective-taking experiences focused on imagined intergroup contact with individuals from marginalized groups can increase prosocial behavior toward them (van Loon, Bailenson, Zaki, Bostick, Willer 2018). Intergroup contact theory hypothesizes that reducing anxiety, which is the cause of increased stereotyping against the outgroup, and permeating the social encounter with positive emotions (Miller, Smith, & Mackie, 2004) leads to prejudice mitigation (Pettigrew & Tropp, 2006). Beyond these insights from social psychology, sparse literature has explored how to design immersive story-worlds to instill prosocial attitudes and behaviors, including the effectiveness of employing photorealistic techniques. This research fills this gap and challenges the assumption that human perceptions inside virtual and physical worlds are equal if digital assets are photorealistic. By creating a taxonomy of design strategies for inclusive VR, displaying data gathered during playtesting six state-of-the-art VR experiences, it identifies which affordances and methodologies are significant to inducing compassion and prosocial attitudes in immersive social encounters and contributes a pragmatic approach to the design of VR for bias mitigation while considering the craft, ethical and humanistic dimensions of the medium.

Modular synthesis is a continually evolving practice. Currently, an eective

taxonomy for analyzing modular synthesizer design does not exist, which is

a signicant barrier for pedagogy and documentation. In this dissertation,

I will dene new taxonomies for modular control, patching strategies, and

panel design. I will also analyze how these taxonomies can be used to in-

uence the design of musical applications outside of hardware, such as my

company Unltered Audio's software products. Finally, I will present Euro

Reakt, my collection of over 140 module designs for the Reaktor Blocks format

and walk through the design process of each.

When a sound source is spatialized over loudspeakers, the source undergoes a series of transformations before it ultimately reaches the listener. The source is first transformed by panning and other spatialization algorithms based on its spatial parameters in what this dissertation calls the virtual field. Next, the source is projected by the loudspeaker array into the acoustic field further transforming the source. The virtual field, loudspeaker array, and acoustic field form the spatial audio field (SAF) which significantly influences how a spatial composition is perceived. This dissertation presents the theory of spatial audio fields and culmination of research and development of new compositional techniques based on manipulation of loudspeaker gain curves in the virtual field. The majority of current spatial techniques focus on manipulation of a source's spatial parameters and parameters of the spatial image. While sources in spatial audio compositions are assigned positions and trajectories that often change throughout the work, the spatialization algorithms of the virtual field are usually fixed, the loudspeaker array does not change, and the acoustic field rarely changes during the performance. Thus the specific SAF used in a given work, whether constructed deliberately as part of the compositional process or constructed arbitrarily, is usually static. The SAF is largely unexplored for new compositional techniques on short musical timescales. As the loudspeakers and acoustic field are physical components of the SAF, changes to each are only practical on larger timescales from musical phrases to form. However, as the virtual field is usually constructed digitally, it can be changed on timescales down to the sample rate. Therefore, the focus of this research is on developing compositional techniques in the virtual field. To carry out this research, an experimental software program was developed that not only allows for realtime spatialization of multiple audio sources, but also allows for realtime changes to the virtual field. This research has led to the development of new compositional elements, methods of synthesis, and spatial extensions to traditional compositional techniques. These methods and techniques are not limited to spatial audio using loudspeakers in a space, but can also be used in spatial audio over headphones where the acoustic field is almost entirely simulated. This opens up possibilities for developing new compositional techniques based on changes to the simulated acoustic field on shorter musical timescales in conjunction with the techniques in the virtual field developed in this research.