Skip to main content
Open Access Publications from the University of California

Digital arts and culture can engage the rich world of human imagination and social construction. This theme explores digital art, cultural production, and theory focused on creating new ideas, worlds, and visions. Cognition no longer refers to only "what is inside the head" - it is now seen as contextual, distributed across individuals and artifacts, and embodied. Relevant topics include: artificial intelligence (AI) and the arts, philosophy of mind, game AI, metaphor and analogy, narrative, poetics, neuroscience and the arts, creative systems, supports for human creativity, language and mind, emotion/affect, and related areas exploring relationships between cognitive science, digital culture, and the human condition.

Theme Leader:
Fox Harrell, Assistant Professor, Digital Media
School of Literature, Communication and Culture, Georgia Institute of Technology
Director of the Imagination, Computation and Expression (ICE) Lab/Studio
Member of Graphics, Visualization, and Usability (GVU) Center

Cover page of Interactive Story Generation for Writers: Lessons Learned from the Wide Ruled Authoring Tool

Interactive Story Generation for Writers: Lessons Learned from the Wide Ruled Authoring Tool


The authoring of interactive, generative narrative is a task that typically requires an extensive multi-disciplinary background in computational and narrative theory. Wide Ruled is an authoring tool that aims to address this problem by providing a friendly, intuitive, story-centric interface to an author-goal driven textbased story planner. Over the past two years, this system has been used repeatedly by technical and non-technical users in multiple classroom settings, and evolved into a widely used and publically available story authoring system. In this work, we describe the successes and failures of Wide Ruled, and how it provides a critical evolutionary step in developing a truly usable, writerfriendly, and practical interactive story authoring environment.

Cover page of Material-Based Imagination: Embodied Cognition in Animated Images

Material-Based Imagination: Embodied Cognition in Animated Images


Drawing upon cognitive science theories of conceptual blending and material anchors, as well as recent neuroscience results regarding mirror neurons, we argue that animated visual graphics, as embodied images whose understanding relies on our perceptual and motor apparati, connect both material and mental notions of images. Animated visual images mobilize a reflective process in which material-based imaginative construction and elaboration can take place. We call this process as “material-based imagination,” in contrast to the general notion of imagination as purely a mental activity. This kind of imagination is pervasive in today’s digitally mediated environments. By analyzing a range of digital artifacts from computer interfaces to digital artworks, we show the important role of imaginative blends of concepts in making multiple levels of meaning, including visceral sensation and metaphorical narrative imagining, to exemplify expressiveness and functionality. The implications of these analyses collectively form a step toward an embodied cognition approach to animation phenomena and toward recentralizing understanding of artistic and humanistic production in cognitive research.

Cover page of Not Me: Collaboration and Co-production with Language Processing Systems

Not Me: Collaboration and Co-production with Language Processing Systems


This paper explores the use of language processing technologies for interactive artwork and studio art production. I consider text in multiple roles: as data, as index, and as a medium for interaction. After describing initial efforts with a dysfunctional chatbot, I discuss my recent work with language processing in the creation of studio art objects, and speculate about the extension of those techniques to address the large corpora of personal media we accumulate online.

Cover page of Comme il Faut: A System for Simulating Social Games Between Autonomous Characters

Comme il Faut: A System for Simulating Social Games Between Autonomous Characters


Modern video games have highly developed computational models of physical space, which allow sophisticated play in the physical realm. However, computational models of social interaction are rare, offer limited social play, and require a large amount of authoring to create. We believe that a computational model of social interaction inspired by appropriate humanities and social science concepts could help alleviate these problems and open up new areas of social play. In this paper, we describe a playable model called Comme il Faut that uses a social artificial intelligence system particularly inspired by Goffman’s dramaturgical analysis and Berne’s psychological games, constructed for authoring power rather than fidelity with the everyday world. Our theoretical basis, the system’s relation to other digital media and games, and its implementation are presented to explain Comme il Faut and our approach to enabling social play.

Cover page of Writing with Complex Type

Writing with Complex Type


Writing practices that integrate dynamic and interactive strategies into the making and reading of digital texts are proliferating as more of our reading experiences are mediated through the screen. In this paper we argue that rarely do current approaches to creating digital texts operate at the basic textual level, that of the letterform itself. We argue further that such neglect is partially the result of the fact that current font technology is based on print paradigms that make it difficult to work programmatically at the level of individual letters. We then discuss work that has been made with software produced in our lab that suggests the creative possibilities in being able to easily specify behaviours at such a level. We conclude by proposing that writers, typographers and programmers start thinking beyond Postscript-like formats such as OpenType or TrueType to collaboratively develop a new ComplexType format (or formats) that is designed for the twentyfirst century as opposed to a simulation of the fifteenth.

Cover page of QuestBrowser: Making Quests Playable with Computer- Assisted Design

QuestBrowser: Making Quests Playable with Computer- Assisted Design


Good gameplay has been characterized as a series of interesting choices. Therefore, to have gameplay of any sort requires the player to be presented with decisions. Given this definition, many quests within computer role-playing games are not playable as they currently exist. Instead, quests are given to the player as a series of tasks to perform in a specific way in order to advance the story within the game. We look at making quests playable – adding choices for the player – and what a system that could support playable quests would look like. Finally, we address the impact playable quests would have on a designer and discuss QuestBrowser, the system we created to handle these concerns.

Cover page of Towards a Critical Technological Fluency: The Confluence of Speculative Design and Community Technology Programs

Towards a Critical Technological Fluency: The Confluence of Speculative Design and Community Technology Programs


In this paper we discuss the use of speculative design as an approach to developing technological fluency. We provide overviews of technological fluency and speculative design, trace their conceptual connections, and then outline the use of a speculative design approach to technology fluency programs, providing an example from a current project. We then conclude by discussing how a speculative design approach can extend the idea of technology fluency towards new directions: broadening common understandings of the practices of technology development and adding a dimension of criticality.

Cover page of The Emotions (after Charles Darwin)

The Emotions (after Charles Darwin)


Rapid changes in science, technology and new media will lead to more sophisticated ideas about what it means to be human, in thought, body, emotional response and artistic expression. New relationships will form between humans, machines and animals with the human functioning as a networked resource that can be accessed globally over the internet.

This paper documents both the technical and theoretical development of the collaborative interactive new media video project “The Emotions (after Charles Darwin)” which explores some of the above concepts. “The Emotions” first tries to establish the existence of the universality of emotions at a biological level, as empirically measured and documented by the results of the control group (non-autistic subjects, as the goal is to document “normal”, i.e. universal emotional response) at the Brain Mind Institute in Switzerland. Secondly, it suggests the potential for subsequent futuristic misuse through genetic and or technological modification (demonstrated by the observer’s ability to interactively modify or transform a given emotion’s video stream at will).

Cover page of Preserving New Media Art: Re–presenting Experience

Preserving New Media Art: Re–presenting Experience


There has been considerable effort over the past 10 years to define methods for preservation, documentation and archive of new media artworks that are characterized variously as ephemeral, performative, immersive, participatory, relational, unstable or technically obsolete. Much new media cultural heritage, consisting of diverse and hybrid art forms such as installation, performance, intervention, activities and events, are accessible to us as information, visual records and other relatively static documents designed to meet the needs of collecting institutions and archives rather than those of artists, students and researchers who want a more affectively vital way of experiencing the artist’s creative intentions. It is therefore imperative to evolve existing preservation strategies for new media art, to include material that provides a rich sense of what it was like to be dynamically present in the live artwork. This paper proposes that simulation strategies with the aesthetic, mechanics and dynamics of the videogame platform, are capable of delivering complex user experiences that can be applied to new media artworks for the purpose of providing users, who may never see the original artwork, with an appreciation of the artists full creative intentions. We further propose that such an approach should be differentiated from the documentation or recording of a particular instance of artwork. We suggest that new creative iterations or versions of new media artworks be designed specifically for this medium through collaboration between artists and game designers and/or developers. Such novel interactive iterations of new media artworks should be conceived so as to enable otherwise temporary artworks to persist in a different form into the future so they may inspire the next generation of artists and inform criticism and research through a direct interactive engagement with the art.

The model of videogames is one marked by diversity and flexibility of means that can exceed the verisimilitude required of simulation or direct mimesis of the live artworks. The creation of a new version of an artwork, which in itself is a construct of the creative imagination, can include virtual worlds and/or mixed reality environments and employ diverse interfaces, emergent behaviours using artificial intelligence and logic engines to fulfill conceptual and exploratory levels of immersion, agency and user experience. The application of videogame design and technology to artworks raises questions of theoretical and practical concern having to do with their desirability as a means of artistic expression, the potential capacity of such re-presentations to deliver affective experience of an artwork as well as their ability of extend the life of new media artworks.