UC Irvine

Human beings have always been interested in understanding themselves and their surround- ings. Learning about the relationship between the two can reveal facts of the present and help predict the future, a critical part to live a better life. With the proliferation of IoT sensor devices, it is now possible to collect quality data for each individual and utilize this data for building personal models that can help to understand the self and environment. However, since this sensor data have different granularities and semantics, the semantic gap becomes even more formidable. Thus, there are challenges in aggregating, integrating, and synchronizing this heterogeneous data to a form such that it effectively describes the life experiences of each individual. In this dissertation, we design a personal chronicle, which contributes a solution to the aforementioned challenges, called Personicle, in which all kinds of personal data streams can be correlated with one another to form a model of a person.

To implement the Personicle, we first attempt to bridge the semantic gap between the low- level multimedia logs and high-level semantics by developing a common daily event model through the data unobtrusively obtained from smart devices. To do this, we define an atomic interval, which brings together the scattered sources of heterogeneous data to partition the data into manageable pieces. This atomic interval lets us segment a day into sequences of similar patterns and use the segments for daily event recognition.

Secondly, we design an event-triggered Ecological Momentary Assessment (EMA) to max- imize the chance of aggregating the semantic data from the users. Unlike the traditional EMA process, which mainly depends on user initiative and intervention, we contribute to overcoming the problems endemic to persistent data collection, such as missing a logging moment or early abandonment, by initiating the EMA process from the system side at the right moment.

Lastly, we propose a fully-automated approach to obtain latent semantic information from all the integrated data aiming to maximize the opportunity of both qualitatively and quan- titatively capturing one’s life experiences. To show a concrete example of this enrichment, we perform an experiment with “Eating” and “Working”, a complex event central to human experiences. These enhanced daily events can then be used to create a personal model that could capture how a person reacts to different stimuli under specific conditions.