Proceedings of the Annual Meeting of the Cognitive Science Society

Recently, cognitive science researchers have revealed that human cognition involves the body and is a kind of self- organization phenomenon emerging from dynamic interaction across body-brain-environment. Some of the data obtained from such cognitive, behavioral, or physiological activities are often complicated in terms of non-stationarity and nonlinearity. Researchers have proposed several analytical tools and frameworks. Recurrence analysis is one of the nonlinear data analyses developed in nonlinear dynamics. It has been applied to various research fields, including cognitive science, for language (categorical) data or motion (continuous) data. However, most previous studies have applied recurrence methods individually to categorical or continuous data. We aimed to integrate these methods to investigate the relationship between speech (categorical) and motion (continuous) directly. To do so, we added temporal information (a time stamp) to categorical data and applied the joint recurrence analysis methods to visualize and quantify speech-motion coordination during a rap performance. Our pilot study suggested the possibility of visualizing and quantifying it.