Uniformly Programmable, Distributed, Reliable, Event-based Systems for Multi-Tier IoT Deployments
With the emergence of the Internet of Things (IoT), data-driven applications have become increasingly powerful. By leveraging a variety of IoT devices and computational resources, developers are able to implement applications to perform a wide range of tasks, from simple data analytics to complex machine learning. While resource-constrained devices and cloud computing have been widely studied in recent years, the rapidly evolving and unique requirements of IoT applications introduce a new set of challenges that inspire lots of discussions and research.Modern IoT deployments often consist of “multi-tier” resources (combinations of distributed sensors, edge devices, and cloud systems). Developers must consider whether to execute computations in remote, resource-rich cloud instances or closer to the sensors and data (at the “edge” of the network) to achieve the desired application performance and operation cost. In addition, modern IoT devices are vastly diverse in hardware capability, network protocols, and reliability. It is increasingly challenging for applications to manage the huge number of components with different characteristics while ensuring scalability, availability, and reliability. To address the above challenges of heterogeneity, programmability, and reliability, we investigate the design of an end-to-end application platform for multi-tier IoT deployment. Our system adopts a publish/subscribe pattern for data exchange and functions-as-a-service programming model to enable event-driven computation. Moreover, it intrinsically integrates data replication using consensus protocol for availability and reliability as well as data and computation locality control. To facilitate analysis, debugging, and reasoning about the highly concurrent applications in the system, we embed a new method to capture the causality of events across distributed multi-tier deployments using a distributed log. Finally, we develop tools that use event causality to repair corrupted data and computation on the fly. We empirically evaluate our system using real-world IoT applications. The results show that our approach significantly simplifies application deployment while improving latency and throughput performance. In addition, it provides a novel way for developers to reason about interdependencies within IoT applications and facilitate failure recovery. The result, we believe, enables us to approach application design with a new event-driven and locality-aware paradigm that is critical for multi-tier IoT environments.