UC Santa Barbara

We live in a world of images and visual data more than ever. Eugène Delacroix, a French artist stated, "[a] picture is nothing but a bridge between the soul of the artist and that of the spectator." As such, creating image-based artwork naturally involves visual communication, and a key type of information included in visual data is the emotions embodied in an image or video.

In affective computing, there exist various interactive systems allowing users to easily express their emotion by adapting in real time to reflect the perceived emotional state of the viewer. For instance, “empathic painting” created by Maria Shugrina is a painter-rendering system that automatically creates digital paintings with parameterized painting features. This system is capable of estimating the viewer’s purposefully displayed emotional state through facial expression recognition.

However, almost no attempt has been made to provide materials for a painting application to allow users to easily express their emotions in their works. As artists use formal elements such as forms and colors to express desired emotions, we need tools that help the users to add emotional conditions for these elements while creating paintings.

Through the introduction of emotion-based brushes, the following research question was explored: how can we incorporate emotions into drawing/painting tools and enable users to control expression? Especially, since immersive virtual reality (VR) environment is effective in producing emotions than less-immersive displays, the application has been designed in the immersive VR environment.

This dissertation examines how the incorporation of emotions into a VR painting application with a provided range of formal elements affects expression and reading of human emotions, and presents two contributions: an analysis of formal elements for expressing emotions, and prototype drawing/painting applications in VR, emoPaint and emoBrush, for incorporating emotion directly as an input.