This dissertation is focused on the development of generative audio systems - a term used describe generative music systems that generate both formal structure and synthesized audio content from the same audio-rate computational process. In other words, a system wherein the synthesis and organizational processes are inseparable and operate at the sample level.

First, a series of generative software systems are described. These systems each employ a different method to create generativity and, though they are not strictly generative audio systems, they lay important groundwork for the rest of the discussion as ideas from and contributions to the fields of generative algorithmic composition, computational aesthetics, music information dynamics, and digital signal processing are introduced.

Second, the dissertation investigates the use of a novel signal processing technique in which time-varying allpass filters are placed into feedback networks, producing synthesis structures capable of yielding interesting emergent sonic behaviors. Ideas from the field of computational aesthetics are employed to allow a large system built from these synthesis structures to become “aesthetically aware.” Many theories about computational aesthetics center around a favorable balance between order and complexity in a stimulus - a successful artistic work is neither too orderly nor too complex. Using a model of human perception based on the “mere exposure” effect, which describes how listener appreciation and boredom change as they experience repeated exposure to a stimulus, the AAS-4 system autonomously determines when and how to modify its own parameters to avoid repetitions that may lead to boredom in listeners.

The dissertation concludes with objective analysis of the generative system by considering the complexity of its output from an information-theoretic perspective. It was found that the generative audio system described here is capable of producing output with equivalent complexity to that of real-world musical examples. It is also shown that the level of complexity in the generated audio and real-world examples falls in-between the low complexity of silence and sinusoids and the maximal complexity of white noise, corresponding with the theories from computational aesthetics. Future directions of this work are also described. Two appendices describing related topics that would disrupt the flow of the dissertation are included.

Musical instruments are tools used to generate sounds for musical expression. Virtual Reality (VR) and Augmented Reality (AR) musical instruments create sounds that may be spatially disjointed from the instrument controls. Spatial audio processing can be used to position the Extended Reality (XR) musical instruments and their corresponding sounds in the same space. This dissertation investigates novel ways of combining spatial reverb models to improve the naturalness of XR musical instruments. Seven spatial reverb systems, combinations of a shoebox spatial reverb model, a raytracing spatial reverb model, and measured directional room impulse response convolution reverb, were compared in a pilot study. A novel hybrid system of synthetic early reflections and directional room impulse responses was preferred for naturalness when tested over headphones with three instruments created by the author: AR electric guitar, AR drumset, and VR Singing Kite. This research culminated in a concert, Spherical Sound Search, which showcased the preferred hybrid system, the three XR musical instruments, and four re-contextualized spatial audio effects (spatial looping, spatial delay, spatial feedback, and spatial compression). The three pieces in the concert explored different aspects of XR modalities and presented the novel system with spatial audio effects to a larger audience by rendering to an octophonic loudspeaker layout.

This Thesis describes the design of the proof-of-concept software application, AudioChalk. AudioChalk, which combines a vector graphics editor with a oscillator bank synthesizer, lets users algorithmically generate, manipulate, or sketch partials by hand in the time frequency domain via intuitive graphical elements. The purpose of developing AudioChalk was to study, from a software developer’s perspective, how content creation software for experimental musicians can be designed to offer both a flexible and efficient workflow. Simultaneously precise and efficient control are gained by sacrificing some of the software’s flexibility. However, if losses of flexibility are carefully considered to not significantly interfere with intended use cases, users often do not perceive a loss of possible functionality. The design choices regarding tradeoffs between efficiency and flexibility that were considered while developing AudioChalk are distilled into a rubric, called idiosyncrasy. Idiosyncrasy as a rubric attempts to generalize the consequences of software design choices from the intended users’ perspective. The intended users in the case of AudioChalk are composers of experimental electronic music.

The two dimensional waveguide mesh is one possible discrete representation

of the wave equation based on the theoretical framework of the one dimensional

waveguide model. The digital waveguide model provides intuitive parametric control

of physical properties, making it conducive to musical synthesis. However,

interesting inharmonic and percussive timbre from the modeling of membranes

and surfaces has often been off limits to real-time synthesis due to the computational

expense of larger two dimensional mesh models.

This thesis covers the results of investigating methods to convert the two dimensional

waveguide mesh into transfer function representations that can be computed

faster in real-time, without compromising the parameters of control that

the physical model provides. Two methods for analyzing the mesh and extracting a transfer function are explored, one conducive to deriving an analytical representation of the mesh, the other a numerical representation. Issues concerning

implementation of analysis and the using the transfer functions as filters are investigated.

Finally, methods for approximating the parametric control of the physical

model in the filter representations are introduced.

This dissertation introduces a physically-informed, abstract synthesis method that applies loopback frequency modulation (FM) to real-time parametric synthesis of percussive sounds. Loopback FM is a variant of FM whereby the output ``loops back'' to modulate its frequency by an amount determined by a feedback coefficient which, when made time-varying, results in pitch glides. Here, loopback FM is used to parametrically synthesize this effect in struck percussion instruments, known to exhibit this characteristic due to nonlinearities in modal coupling. Inspired by the sonic potential of physics-based nonlinear percussion synthesis models, the loopback FM synthesis method uses an abstract synthesis approach in order to create a wide variety of percussive sounds in real-time. A linear, modal synthesis percussion model is modified to use loopback FM oscillators, which allows the model to create unique, eclectic, and experimental sounding percussive hits in real-time. Musically intuitive parameters are emphasized resulting in a usable percussion sound synthesis method.