Organizations collect a substantial amount of user' data from multiple sources to explore such data analytically and derive meaningful insights. One of the obstacles that prevent organizations from reaping the benefits of the analysis task is the low quality of the previously collected data. Hence, most of the data preparation time is dedicated to cleaning the data from fixing type errors to removing the uncertainty or ambiguity of some data using data cleaning techniques. A new paradigm for handling such issues is integrating the cleaning process within the query execution workflow to clean the needed tuples rather than performing the cleaning step prior to query execution on the entire dataset. In this thesis, we tackle the challenge of applying the query-driven cleaning approach in the case of probabilistic queries.First, we present TQEL, a framework that integrates the entity linking task with query processing to answer top-k entities' queries on top of a collection of social media blogs. The entity linking process removes the ambiguity of certain words in any textual snippet by linking such words to real-world entities. The TQEL framework offers two variants: TQEL-exact and TQEL-approximate, that retrieve the exact/approximate top-k results. TQEL-approximate, using a weaker stopping condition, achieves significantly improved performance (with the fraction of the cost of TQEL-exact) while providing strong probabilistic guarantees (over two orders of magnitude lower EL calls with a 95% confidence threshold compared to TQEL-exact). Subsequently, we propose TQELX, a framework that generalizes the previous approach to support multiple aggregation functions and other group-based aggregation queries. TQELX is an analysis-aware cleaning for probabilistic queries that use the approximate confidence computation technique. TQELX tightly incorporates the cleaning step in multiple stages of the Monte-Carlo simulation execution to return the results as quickly as possible. We compare our approach against multiple probabilistic query answering baselines and show that TQELX outperformed them in total execution times. Lastly, we discuss the incremental view maintenance problem in probabilistic databases and provide a solution to speed up the execution process in the case of database updates. Naively, if a tuple is inserted, deleted or updated, the previously computed results become obsolete, requiring the query's re-execution. In cases where the query uses approximate confidence computation techniques, the overhead of such process incurs overheads and unacceptably delays the overall execution experience. We implement PIVM, a solution built on top of PostgreSQL to support delta computation techniques efficiently. Our experiments demonstrate the effectiveness of using such an approach on multiple select-project-join queries and report that PIVM offers a massive execution speed-up in the case of updates.

In the era of real time big data stream processing, the past decade has witnessed emergence of wide variety of applications to tap into live social data feed to gain awareness about events, opinions and sentiments of the community. Social data has the potential to bring new functionalities and improvements in a wide variety of domains from Emergency Response to Political Analysis. Thus, there is an increasing demand for executing up-to-the-minute analysis tasks on top of these dynamic data sources by modern applications. Such new requirements have created new challenges for traditional data processing techniques. In this thesis, we respond to some of these challenges.

First, we explore the problem of online adaptive topic focused tweet acquisition. Specifically, we propose a Tweet Acquisition System (TAS), that iteratively selects phrases to track according to a reinforcement learning algorithm. The selection follows an explore-exploit policy to approximate the effectiveness of different phrases in retrieving relevant tweets based on Bayesian inferences. We also develop a tweet relevance model, which enables checking the relevance of collected tweets to the topic of interest based on multiple criteria. The objective of TAS is to improve the recall of collected relevant tweets. Our experimental studies show significant improvements over the state-of-the-art, furthermore the performance gap increases when the topics are more specific.

Subsequently, efficient processing of top-k mentioned entities query posed on a stream of tweets has become a key part of a broad class of real-time applications, ranging from content search to marketing. Given that words are often ambiguous, entity linking becomes an important step towards answering such queries. Furthermore, the continuous and fast generation of tweets makes it crucial for such applications to process those queries at an equally fast pace. In order to address these requirements, we propose TkET (pronounced ticket) as an analysis-aware entity linking framework for efficiently answering top-k entities query over Twitter stream in an sliding window fashion. The comprehensive empirical evaluation of the proposed solution demonstrates its significant advantage in terms of efficiency over the traditional techniques for the given problem settings.

The world today runs on data and lots of private data, whose security is of paramount importance. We have many secured cryptographic techniques which are practical but not completely secure. On the other hand we have completely secure techniques which are not practical to be run as far as time complexity is concerned. In this scenario, we would like to exploit the fact that there is partial sensitivity of data. We partition the data into sensitive and non sensitive parts with the former being expensive to retrieve while the later is comparatively inexpensive. We propose that combining both the costs using our scheme gives better result than considering the entire data as sensitive. We will prove this using mathematical models as well as by experiments.

Unprecedented growth in sensing, data capture devices, communication, and computing technologies have enabled us to capture and analyze almost every aspect of our lives. Such a prospect has resulted in the development of a myriad of applications in the cyber-physical systems, and the Internet of Things domain. While limitless possibilities exist, their realization has led to the grand challenge of designing a new class of systems that can provideseamless support for efficient and easier to program sensor-based applications. Furthermore, as sensor data processing often consists of complex compiled code, expensive machine learning, and signal processing code, systems that can scale to data generated by a myriad of interconnected sensors and devices embedded in the environment are needed.

This thesis presents TippersDB, a middleware system designed to build sensor-based smart space analytical applications. The intended goal of TippersDB is to ease the task of developing complex smart space applications, to develop an extensible system that allows diverse/heterogeneous sensors (with possibly overlapping functionalities) to co-exist and to be seamlessly integrated into the system without requiring application redesign, and to exploit the semantics of the physical world, as well as, sensor capabilities to optimize the applications. TippersDB provides a powerful data model that decouples semantic data about the application domain from sensor data using which the semantic data is derived. By supporting mechanisms to translate data, concepts, and queries between the two levels, TippersDB relieves the application developers from having to know or reason about either the type or location of sensors or write sensor specific code. In addition, it allows for multiple optimizations based on smart space semantics to improve query processing.

In particular, in this thesis, first we introduce the SmartBench benchmark that we developed to analyze the existing database technologies in terms of their strength/weaknesses and suitability in supporting IoT applications. Next, we introduce the TippersDB data model and describe its query-driven translation of sensor data. We then provide a summary of the system implementation and show a performance evaluation of TippersDB using IoT benchmark queries. This is followed by the description of the progressive query processing techniques used in TippersDB to reduce wait time for the users by providing them with early results. Finally, we highlight the benefits of TippersDB through a case study.

In the era of big data, in addition to large local repositories and data warehouses, today’s enterprises have access to a very large amount of diverse data sources, including web data repositories, continuously generated sensory data, social media posts, clickstream data from web portals, audio/video data capture, and so on. As a result, there is an increasing demand for executing up-to-the-minute analysis tasks on top of these dynamic and/or heterogeneous data sources by modern applications. Such new requirements have created challenging new problems for traditional entity resolution, and data cleaning in general, techniques. In this thesis, we respond to some of these challenges by developing an analysis-aware approach to entity resolution.

First, we explore the problem of analysis-aware data cleaning in the context of selection queries. Specifically, we propose an “on-the-fly” data cleaning framework in the context of SQL-like selection queries. The objective of this framework is to perform the minimal number of cleaning steps that are required to answer a user query correctly. Our approach leverages the concept of vestigiality to reduce cleaning overhead. We conducted a comprehensive empirical evaluation of the proposed solution to demonstrate its significant advantage in terms of efficiency over the traditional techniques for the given problem settings.

Subsequently, we study analysis-aware data cleaning for the more general case where queries can be complex SQL-style selections and joins. In particular, we develop a framework for integrating entity resolution techniques with query processing. The aim of this framework is to utilize the query semantics to reap the benefits of early predicate evaluation while still minimizing redundant computation in the form of data cleaning. This framework relies on the notion of polymorphic operators, which are analogous to the common relational algebra operators with one exception: they know how to test the query predicates on the dirty data prior to cleaning it. We conducted extensive experiments to evaluate the effectiveness of our approach on real and synthetic datasets.

Overall, our experiments demonstrate outstanding results – that is our analysis-aware approaches are significantly better compared to traditional ER techniques, especially when the query is very selective.

Social media, web search logs and online purchases are only some of the sources used my private and public organizations to collect information about individuals.

While, the aggregated data are valuable for research and commercial use, they can pose a direct threat to a user's privacy.

The field of privacy preserving data sharing has emerged, in order to create tools, that can enable institutions and companies to shares their clients' information, while protecting their privacy.

Differential privacy (DP) has been recognized as the de-facto privacy framework for interacting with private information.

In this thesis, we continue the work on release histograms with formal privacy guarantees.

Particularly, we investigate the effect of sorting as a technique for improving the accuracy of the final approximation.

We identify the right settings to use it, and when to avoid it.

DP aims to protect users' records from every possible type of inference attack.

As a result, the DP is a very pessimistic privacy framework that considers every piece of information as sensitive.

In the context of this thesis, we will relax the previous requirement.

We propose one-sided privacy (OSP) a novel privacy framework that is able to handle data that can be classified as sensitive or non-sensitive.

Our empirical results show that OSP can support new types of application, and offers meaningful utility, in cases that DP is known to perform poorly.

New technology domains, such as the Internet of Things (IoT), are adding a large number of new devices with Internet connectivity to the spaces where we work and live. These devices are accelerating the collection of user data at an unprecedented rate. On the other hand, new data privacy regulations are emerging all around the globe to protect people's privacy (such as the California Consumer Privacy Act CCPA, European General Data Protection Regulation GDPR, and Brazilian LGPD, among many others). These regulations have put forward stringent requirements on organizations with respect to what should be done when user data is handled. Organizations have been scrambling to adapt their infrastructure in response to these regulations and many have been punished with hefty fines for improper handling of data.

Data Management Systems are at the core of organizations collecting such data. They handle its capture, retention, processing, and sharing. To protect people's privacy, Data Management Systems require, among others, to be able to enforce individuals' privacy preferences. These issues become even more challenging given the scale at which data is captured in new domains such as the IoT. This thesis presents various solutions to support fine-grained privacy policies for data protection when dealing with upcoming IoT applications.

In particular, this thesis outlines framework to enable people to communicate their privacy preferences/policies to smart spaces to address the challenge of Policy-based Privacy-by-design Smart Spaces. This includes a language which allows users to define who, and under which circumstances, can access their data collected by IoT systems. Supporting the definition of such user-defined fine-grained IoT policies in Data Management Systems can lead to scenarios where a large number of them have to be enforced in real-time. The thesis presents a system to answer queries efficiently while enforcing a very large number (hundreds of thousands) of user policies to address the challenge of Scalability of Policy Enforcement. In modern DBMS and particularly in IoT settings, data exists at different semantic levels where dependencies capture the constraints that exist within the data. This thesis describes an approach to prevent leakages through various different dependencies on access controlled data.

The prototypes built as part of the solutions for the above challenges have been integrated into two IoT Systems deployed at UC Irvine. The first is an IoT test bed entitled TIPPERS and the second is privacy preserving middleware called PE-IoT. These integrations show the feasibility of the approaches presented to specify and efficiently enforce privacy policies for supporting IoT applications.

In this thesis, we propose EnrichDB, a new DBMS technology designed for emerging domains (e.g., social media analytics and sensor-driven smart spaces) that require incoming data to be enriched using expensive machine learning/signal processing functions prior to its usage. To support online processing, today, such enrichment is performed outside of the database as a static data processing workflow prior to its ingestion to the database. Such a strategy could result in a significant delay from the time when data arrives and when it is enriched and ingested into the DBMS, especially when the enrichment complexity is high. Also, enrichment at ingestion could result in wastage of resources if applications do not use/require all data to be enriched. EnrichDB’s design represents a significant departure from the above, where we explore seamless integration of data enrichment all through the data processing pipeline --- at ingestion, triggered based on events in the background, and progressively during query processing. The cornerstone of EnrichDB is a powerful enrichment data and query model that encapsulates enrichment as an operator inside a DBMS enabling it to co-optimize enrichment with query processing. The first chapter of the thesis describes this data model of the system.

In the second chapter of the thesis, we describe two implementations of the EnrichDB system. In the first implementation, we have taken a middleware-based approach where the database management system is treated as a black-box and enrichment is performed in a separate server, called the enrichment server. This is a simpler and portable implementation, since a user can utilize any DBMS as the storage system for storing the underlying data objects with a small code change. In the second implementation, we describe a layered implementation on top of PostgreSQL, where we have used the extensibility features of PostgreSQL to perform enrichment efficiently closer to the data where it resides. We used user-defined functions, stored procedures, indexes, and incremental materialized views to perform data enrichment and produce query results efficiently.

In the third part of my thesis, we present the algorithms implemented in above systems that can optimize enrichment with query processing. We have chosen the quality metric of F1-measure as the quality of set-based queries. Improving query results gradually by choosing online samples were explored in Online Approximate Query Processing systems. However, such systems did not consider data enrichment to improve the quality of query results and the underlying data was considered to be static. The goal of data enrichment in the EnrichDB system is to enrich objects in a way that can produce good quality results as soon as possible, formalized as progressive score. Experimental results on real world datasets and queries show that the algorithm performs significantly better than the traditional sampling based approaches.

The explosion of massive media data induced by the proliferation of digital cameras, mobile devices as well as the emergence of online media websites, has led us into the era of big data. Accurate and effective analyses of the big multimedia data to support semantically enriched representation in terms of events, activities, and entities can bring transformative improvements to a variety of application domains. The basic form of multimedia analysis for more sophisticated interpretation is characterized by questions such as ``who, what, where, when" that identify subjects, activities, locations, and time associated with images/video segments. In this thesis, we primarily focus on the ``who" question, which is referred as the person identification problem in multimedia data.

While advances in image processing and computer vision has resulted in powerful techniques for person identification, such techniques based on the facial appearance representations, are usually prone to errors due to a variety of factors including noise, poor signal quality, occlusion, etc. It is widely recognized in the multimedia research community that additional contextual features can be leveraged to bring significant improvements to such tasks. Nevertheless, how to systematically utilize the heterogeneous contextual information still poses a big challenge. Besides, the person identification procedure is conventionally processed in an ``offline" setting where the typical goal is to achieve complete annotations of the whole collection before further applications. Such an ``offline " process is not tenable when dealing with big multimedia data, since the limitation of computational resources as well as restriction of manpower does not allow us to process every image with each possible tag and clean up every potentially noisy result.

We note that similar challenges also arise in the database domain, especially for the entity resolution task. To address these challenges, recent entity resolution research has explored a series of powerful methods including techniques to exploit relationships, contextual information, domain semantics in the form of constraints and ontologies, etc. for the purpose of resolving references in structured/semi-structured and unstructured textual data. Additionally, query-driven data cleaning techniques have also been proposed and explored to resolve the challenges of big data.

In this thesis, we aim to explore how such advanced entity resolution techniques can be exploited to improve semantic interpretation of multimedia data, specifically for the person identification problem. We first explore how to leverage the automatic data cleaning techniques to exploit relationships, contextual information, domain semantics, constraints, etc., to enhance the performance of face clustering and recognition. Then, we propose the new paradigm for face clustering/tagging suited for big data where image enrichment is seamlessly integrated into the image retrieval/analysis process -- we refer to this new paradigm as ``query-driven image enrichment".

Particularly, we first study the person identification problem in the context of surveillance videos and propose a context-based framework for low-quality data, which integrates multiple contextual information leveraging the entity resolution framework called RelDC to improve the performance of person identification. Inspired by the significant results improvement, we investigate the face clustering problem and propose a unified framework that employs bootstrapping to automatically learn adaptive rules to integrate heterogeneous context information together. Furthermore, we exploit the human-in-the-loop mechanism to leverage human interaction to achieve high quality clustering results. Later, to address the challenges of big multimedia data, we propose the query-driven approach to face clustering/tagging which investigates the query-driven active learning strategies in order to achieve the accurate query answers with minimum user participation.

In this thesis, we study privacy in the context of Decision Support(DS) applications. DS applications utilize data collected from numerous sources to guide important decisions. However, such applications could face severe privacy challenges if the data contains sensitive information about individuals. While techniques such as differential privacy are suited for privacy-preserving data sharing, their usefulness in the context of decision support (DS) applications is limited due to privacy and utility trade-offs as these techniques do not offer any guarantees on the quality of results. DS tasks, in contrast, require guarantees on the output quality to avoid making misleading and inaccurate decisions.

We explore the concept of minimally invasive data exploration for decision support that attempts to minimize privacy loss while supporting bounded guarantees on accuracy. We build a demo application and conduct empirical studies to understand privacy utility trade-offs of different privacy techniques and to highlight the need for accuracy driven privacy preserving data analysis. We formally define decision support queries and their accuracy requirements and present privacy preserving algorithms to answer these queries that minimize the privacy loss while providing the required accuracy guarantees for decision support.