- Main
Intelligence Beyond the Edge in IoT
- Yu, Xiaofan
- Advisor(s): Rosing, Tajana
Abstract
The unprecedented developments in machine learning and hardware technologies are bringing a new wave of deploying intelligent sensor systems at the edge for Internet of Things (IoT) applications. However, significant gaps remain in realizing edge intelligence that is scalable across large networks, adaptable over time, interactive with humans via natural language, and energy-efficient on resource-constrained devices, limiting its potential for real-world deployment.In this thesis, I present three novel end-to-end systems designed to bridge these gaps. All systems utilize co-design approaches that integrate embedded systems, AI algorithms, and IoT applications. Together, the proposed systems lay the foundation for future edge intelligence, enhancing the practicality of intelligent edge solutions across a wide range of applications.
The first system, Async-HFL, tackles the scalability and efficiency challenge of edge learning in large-scale IoT networks that have thousands of nodes. Async-HFL is a novel asynchronous federated learning framework for hierarchical IoT networks, allowing asynchronous model aggregations and adaptive topology management in response to varying computational resources, network bandwidth, and device failures.The second system, LifeHD, addresses the lifelong adaptability and efficiency challenge of edge intelligence, allowing individual edge systems to continuously adapt to new environments without prior knowledge nor supervision. LifeHD achieves on-device lifelong and unsupervised learning through a two-tier memory architecture based on Hyperdimensional Computing (HDC). HDC is a novel brain-inspired computing paradigm that excels in lightweight on-device learning and is robust against catastrophic forgetting. The third system, SensorChat, focuses on natural language interaction with sensor systems at the edge. SensorChat allows users to ask practical questions about their daily activities monitored by the sensors, and responds with accurate answers derived from long-term multimodal sensor data. SensorChat utilizes a new three-stage design powered by Large Language Models and an explicit sensor data query module, achieving average response times of 2.3 seconds in the cloud and 10.0 seconds on an NVIDIA Jetson Orin.
In the future, a more comprehensive approach to designing intelligent edge systems should be explored, with promising directions such as embodied physical edge intelligence, near- or in-sensor computing, and sustainable computing.