The ongoing revolution in the mobile networked system (consisting of mobile clients, and the 4G/5G network infrastructure) is reaching a critical stage. On one hand, it has been operational for years, resulting in billions of users and tens of petabytes per month for mobile data traffic. On the other hand, users today regularly complain about network failures, unsatisfactory performance, and security threats with alarming frequency. The impact of such problematic issues may further exaggerate, as the emerging next-generation applications (such as interactive virtual/augmented reality, self-driving cars, telemedicine, and drone-based delivery, etc.) pose more stringent requirements. The convention wisdom seems to attribute most of these issues to the poor wireless channel quality, thus motivating a range of new access technologies to mitigate them.

This dissertation demonstrates that, besides the wireless link, the architectural limitation should share equal responsibility. The fundamental problem is that, the entire networked system does not possess sufficient intelligence on what problems may arise, why such issues occur under the given scenarios, and how to react. The mobile clients lack runtime information on the underlying “black-box” network operations, whereas the infrastructure suffers from the complex interplays of protocol functions among distributed nodes. Both again are rooted in the basic design tenet of “smart core, dumb terminal” adopted by the current mobile networked systems. Our study has uncovered a range of real issues incurred by the architectural limitations (rather than the poor wireless link), including handoff instability, suboptimal roaming, and long data access latency perceived by both the client and the infrastructure. Most of the current solutions are piecemeal efforts without looking into the core system architecture designs.

This dissertation thus explores a new dimension in mobile networked systems. We seek to augment the system intelligence on verifying the baseline designs a priori, detecting problematic scenarios based on runtime information feedback, understanding their root causes, and taking smart actions. To this end, we propose a new knowledge plane for the system software stack, which offers a novel primitive for the mobile networked system that helps to unleash its architectural limitations. The knowledge plane follows the “smart client, simple infrastructure” principle but leaps one step forward. On one hand, it explores the “data-driven approach to smart clients” by exposing rich network information at runtime and leveraging some recent results on data sciences and machine learning. On the other hand, it seeks to devise simple, yet verifiable protocol solutions with provable properties by applying techniques on network verification and distributed computing. In this way, we enable a simpler and more open networked system for the mobile devices.

The main contribution of this dissertation is the design, instantiation, analysis, and validation of the knowledge plane and its benefits on performance and reliability. The concrete results cover on both the client device and the network infrastructure. On the mobile client side, we construct the knowledge plane using the “bottom-up” data-driven system design, by enabling the client-side access to rich network data at runtime. It thus opens access to the typically “closed” network operations without infrastructure changes. We build MobileInsight, the first tool that opens up the runtime, fine-grained cellular network data and offers protocol analytics (using AI-based inference) on commodity phones; it has been used by 247 universities and companies during its first-year release so far. As a showcase application, we develop iCellular, an enhanced client-centric, multi-carrier roaming service. By leveraging low-level network data analytics, iCellular boosts the device with up to 3.74� throughput improvement, 1.9� latency reduction over the state-of-the-art solution from Google Project Fi.

On the infrastructure side, we enable the knowledge plane with the “top-down” approach. We treat the infrastructure as a distributed system, define the structural properties (stability, availability, consistency, etc.) that capture the high- level demands, apply verification and distributed computing techniques to reason about them, and enforce provable reliability and efficiency. At the management plane, we conduct the first study on the stability of the distributed mobility management. We show that, policy/configuration conflicts exist in reality, and force the device to oscillate among base stations permanently. We prove the necessary/sufficient conditions for the stability, and create MMDiag that detects and resolves the policy/configuration conflicts. At the control plane, we build DPCM, the first paradigm in the mobile network that parallelizes the control plane procedures for low-latency data access. It is inspired by the generalized CAP theorem, and leverages the device-side state replica achieve 2.1x–11.5x latency reduction on average in different scenarios.

These results show that, augmenting the future mobile networked system with intelligence can benefit both the client and the infrastructure. The knowledge plane presented in this dissertation provides a viable solution to move one step closer toward a future mobile networked system (5G and beyond) with "Intelligence-as-a-Service".