The interest in building blockchain Decentralized Applications (DApps) has been growing over the past few years. DApps are implemented as smart contracts which are programs that are maintained by a blockchain network. Building DApps, however, faces many challenges—most notably the performance and monetary overhead of writing to blockchain smart contracts. To overcome this challenge, many DApp developers have explored utilizing off-chain resources—nodes outside of the blockchain network—to offload part of the processing and storage.In this paper, we propose RollStore, a data indexing solution for hybrid onchain-offchain DApps. RollStore provides efficiency in terms of reduced cost and latency, as well as security in terms of tolerating byzantine (i.e., malicious) off-chain nodes. RollStore achieves this by: (1) a three-stage commitment strategy where each stage represents a point in a performance-security trade-off—i.e., the first stage is fast but less secure while the last stage is slower but more secure. (2) utilizing zero-knowledge (zk) proofs to enable the on-chain smart contract to verify off-chain operations with a small cost. (3) Combining Log-Structured Merge (LSM) trees and Merkle Mountain Range (MMR) trees to efficiently enable both access and verification of indexed data. We experimentally evaluate the cost and performance benefits of RollStore while comparing it with BlockchainDB and BigChainDB.

Emerging edge applications such as the Internet of Things (IoT) and Industry 4.0

require stringent real-time guarantees. This makes it infeasible to rely solely on faraway

cloud nodes for these applications. Similarly, edge machines are less capable and

reliable than cloud machines. To balance the trade-off of edge and cloud computing,

we propose a data infrastructure that spans both edge and cloud machines, that we call

edge-cloud computing. We tackle one of the fundamental data management challenges

in edge-cloud computing, the problem of data indexing. We propose Cooperative LSM

[3] (CooLSM), a distributed Log-Structured Merge Tree that is designed to overcome

the unique challenges of edge-cloud indexing such as machine and workload

heterogeneity and the communication latency asymmetry between the edge and the

cloud. To tackle these challenges, CooLSM deconstructs the structure of LSM into its

basic parts. This deconstruction allows a better distribution and placement of resources

across edge and cloud devices. For example, append-specific functionality is managed

at the edge to ensure appending and serving data in real-time, whereas resource-intensive

operations such as compaction are managed at the cloud where more

computing resources are available.

Access to user mobility and activity data (uMAD) is crucial for researchers and practitioners in various areas of technology and infrastructure planning. It reveals a number of aspects of user behavior and trends at different spatio-temporal scales which in turn provide invaluable information to guide the design, operation, and management of critical infrastructure, services and applications. However, previous academic/industry efforts to collect user mobility and activity (uMA) information face important challenges raised by issues such as uMA data diversity, privacy and protection concerns. Consequently, even if uMA data is collected successfully, it cannot be generalized and/or shared publicly. To address these challenges, there has been significant work on the generation of synthetic uMA datasets as well as work on data anonymization. Prior work in these areas, however,target specific applications and datasets, and thus make it harder to generalize them for use across different scenarios.

Our aim is to fill these gaps by providing an uMA ecosystem that manages classification, generation, evaluation and analysis. As part of this goal, our pipeline uMAD aims to include the following features: Classification: enabling existing or new uMA data to be classified into our proposed taxonomy buckets; Generation: allowing users to capture patterns and generate realistic uMA datasets by leveraging well known Machine learning generation models like Generative Adversarial Networks (GANs); Trace Analysis: helping users analyze and visualize patterns in existing and new uMA datasets; and Model Analysis providing users with a broad understanding of the ML model resource consumption and parameters. uMAD’s open source a command line interface (CLI) is eventually meant to generate realistic synthetic uMA datasets that mimic existing traces for a range of user-configurable parameters and provide users with existing datasets that can be selected based on the users’ specific needs.

Global-scale data management(GSDM) empowers systems by providing higher levels of fault-tolerance, read availability, and efficiency in utilizing cloud resources. This has led to the emergence of global-scale data management and event processing. However, the Wide-Area Network (WAN) latency separating datacenters is orders of magnitude larger than typical network latencies, and this requires a reevaluation of many of the traditional design trade-offs of data management systems. Therefore, data management problems must be revisited to account for the new design space.

In this dissertation, we propose theoretical foundations to understand the limits imposed by WAN latency on GSDM, and propose practical systems and protocols to minimize the overhead caused by WAN latency. The presented work spans global-scale transaction processing, communication, analytics, and machine learning. In all these directions, the focus is on the trade-off between consistency and latency, where we ask the question: what is the best performance (often latency) we can achieve without compromising the consistency and integrity of data? For transaction processing, we propose a lower-bound formulation for transaction latency that is imposed by the WAN latency. Also, we propose a new paradigm for transaction processing (proactive coordination) that inspired out two proposed protocols, Message Futures and Helios, which can achieve the lower-bound latency. We also propose a communication framework, called Chariots, to scale multi-datacenter communication. Chariots is carefully designed to allow scaling communication while providing a consistent view of the communicated information. Finally, we explore challenges in global-scale analytics and machine learning. Specifically, we propose Ogre, a scalable system for global-scale heterogeneous transactional and analytics workloads. Also, we propose COP, a system designed to speed up machine learning on globally generated data.

State machine replication protocols, like MultiPaxos and Raft, are at the heart of numerous distributed systems. To tol- erate machine failures, these protocols must replace failed machines with new machines, a process known as reconfigu- ration. Reconfiguration has become increasingly important over time as the need for frequent reconfiguration has grown. Despite this, reconfiguration has largely been neglected in the literature. In this paper, we present Matchmaker Paxos and Matchmaker MultiPaxos, a reconfigurable consensus and state machine replication protocol respectively. Our protocols can perform a reconfiguration with little to no impact on the latency or throughput of command processing; they can per- form a reconfiguration in a few milliseconds; and they present a framework that can be generalized to other replication pro- tocols in a way that previous reconfiguration techniques can not. We provide proofs of correctness for the protocols and optimizations, and present empirical results from an open source implementation showing that throughput and latency do not change significantly during a reconfiguration.