UC San Diego

The past decade has witnessed a significant growth of deployed sensors in our daily life, covering applications from healthcare, climate modeling to home automation and robotics. These sensors collect abundant time series data, which facilitate our understanding of various real-life processes. However, the inherent instability of these sensors, combined with the dynamic environments in which they operate, often results in the collection of data that is noisy, sparse, insufficient and varied. This presents a significant contrast to the data typically encountered in other domains such as computer vision and natural language processing. Consequently, current data-driven models trained under controlled experimental settings often fall short of their value in accurate inferencing and analyses.

To address these limitations, we propose to bridge the inherent physical contextual knowledge and external semantic contextual knowledge of sensor time series to build a more robust analysis framework for sensor data. Specifically, we first exploit the inherent physics principles underlying time series data — ranging from mathematical models, spatio-temporal correlations to spectral properties — as inherent contextual knowledge. We leverage such physics knowledge to refine raw sensor time series and enhance data quality through denoising, imputation and augmentation. Additionally, sensor time series are often accompanied with external label names or metadata presented as text. Therefore, we build upon the recent advances in language modeling, to incorporate external semantic contextual knowledge from large language models or pre-train our own domain-specific foundation models. Such semantic knowledge further enriches sensor time series understanding and increases cross-domain robustness. Our framework is robust and demonstrates state-of-the-art performance in multiple tasks such as recognition, forecasting and navigation across sensing systems of various scales, from small-scale personal healthcare monitoring, smart home automation, to large-scale smart building control, energy management, climate modeling and beyond.