Generating acoustic signals to achieve both referential and aesthetic goals
Abstract
The human ability to convey meaning through sound extends beyond spoken language, encompassing the use of tools like musical instruments. People use such instruments to express not only thoughts, but also feelings. How might someone choose to convey such complex meanings with some sounds instead of others? In this study, participants (N=256) used a novel digital musical instrument to produce sound effects to accompany several video clips, such that someone else could match them up later (Referential). Half of these participants were further encouraged to make their sound effects pleasing, towards revealing what distinguishes sounds meant to evoke a particular feeling (Pleasing). The two groups produced different kinds of sound effects, with Pleasing participants favoring sound effects that spanned a narrower range of pitches, while Referential participants created more consistent cross-modal mappings. Taken together, these findings highlight how readily people can convey complex meanings through sound.