Cloud gaming architecture has been proposed and deployed in large-scale com- mercial systems in recent years. It renders and encodes video views on cloud servers, with the resulting video streamed through network to end devices. This approach has the advantage of relieving high computation, power and storage requirements of gaming from end devices. The challenge therefore shifts to streaming high quality video with low latency through fluctuating network. In this thesis, we extend the basic cloud gaming architecture in three aspects. First, we consider not only gaming as target application, but also consider much more virtual immersive applications, such as virtual classroom and virtual art gallery, etc. Second, with the development of ubiquitous wireless network for mobile devices, we specifically consider streaming the video through mobile wireless network. Third, with the growing popularity of mobile auto-stereoscopic 3D displays, we consider capture, record and stream two views (left and right views) in the virtual world so that it can render 3D views on mobile 3D displays. To conclude, we term our extended architecture as cloud mobile 3D display virtual immersive application architecture. Because 3D videos contain two views that doubles video bit rate requirement for the same quality versus 2D video, and also because mobile wireless network is much more fluctuating than wired and WiFi network, streaming high quality 3D video with low latency becomes much more challenging. In this thesis, based on prior research results that show human brain can compensate perceived video quality automatically when one of the views is of inferior quality, we propose asymmetric rendering where we tune graphics rendering quality to be asymmetric in rendering engine. We develop both user experience models and bit rate models by applying different rendering settings and conducting subjective tests. We further develop optimization algorithms which use the above two models to automatically decide the optimal rendering settings for left view and right view to ensure the best user experience given the network conditions. Experiments conducted using real 4G-LTE network profiles on commercial cloud service with different genres of applications demonstrate significant improvement in user experience when the proposed technology is applied.

Over the past few years, there has been an increased number of media applications that have migrated to the cloud, which will enable mobile users to engage in rich media experiences from any mobile platform. In this research, we develop techniques to model and enhance the quality of service for two cloud mobile media applications: Cloud Mobile Rendering (CMR) application and HTTP-based adaptive video streaming.

In CMR application, compute intensive graphic rendering is performed on cloud servers, and the rendered video is encoded and transmitted through wireless network to mobile devices. Although promising to address the computation and battery limits of mobile devices, this approach suffers from challenges posed by limited and unstable wireless bandwidth, such that it is difficult to ensure the quality of service. In this research, we first develop a model to quantitatively measure the user experience of CMR application, considering rendering factors, encoding factors and network factors simultaneously. Secondly we derive several techniques to enhance the quality of service, including a)an adaptive rendering technique which dynamically select the optimal rendering factor based on network condition; b) a prioritized encoding technique which allocates bits among different regions of video frame depending on their priorities; c) an encoding acceleration technique which utilizes rendering information to reduce the computation complexity of video encoding while maintaining the video quality.

In HTTP-based adaptive video streaming, video content are split into a sequence of small segments, which are encoded into several versions with different bit rates and quality, and streamed according to the requests from streaming client. The version of video segments will keep varying according to the available bandwidth. Flucating bandwidth will lead to temporal artifacts such as video rebuffering (freezing), and spatial artifacts such as visual quality variation. In this thesis, we develop the first user experience model for this application through extensive subjective experiments. We validate with high accuracy the user experience model, and demonstrates its applicability to video with different length and motion characteristics.

The effectiveness of traditional physical therapy may be limited by the sparsity of time a patient can spend with the physical therapist (PT) and the inherent difficulty of self-training given the paper/figure/video instructions provided to the patient with no way to monitor and ensure compliance with the instructions. In this dissertation, we propose a virtual PT system using sensors and AI to enable on-demand physical therapy training and balance evaluation at home. This work can be divided into three stages. Firstly, we have developed a cloud-based monitoring and guidance system for home-based physical therapy training. We use a motion capture sensor to track the patient's performance and develop algorithms to address the latency problems in evaluating the patient's performance. Different types of guidance have been designed to help the patient improve the performance. The proposed system is a generalized model that can be applied to many types of diseases, as well as fitness training, ergonomics training, etc. Secondly, we focus on patients with Parkinson's disease (PD) and propose an action understanding, assessment, and task recommendation system. The proposed system is able to understand the patient's movements and identify the movement error. In addition, the proposed system provides personalized task recommendations for the patients. The task recommendations can be fully automated, or if desired, the system may require remote supervision and approval by the PT. Thirdly, we propose an automated balance evaluation system using multiple sensors to enable on-demand balance evaluation at home. The proposed balance evaluation model is able to provide a quantified balance level that is consistent with the human PT’s assessments in traditional balance evaluation tests. To train and validate the proposed systems, we have collected real patient data from the clinic. Experimental results show high accuracy of the proposed systems. By using inexpensive sensors and AI, the proposed virtual PT and balance evaluation system has the potential of enabling on-demand virtual care and significantly reducing cost for both patients and care providers.

Triggered by several head-mounted display (HMD) devices that have come to the market in recent years, such as Oculus Rift, HTC Vive, and Samsung Gear VR, significant interest has developed in virtual reality (VR) systems, experiences and applications. However, the current HMD devices are either tethered with PC/console, or rendering locally on itself (quite clunky to wear), negatively affecting user experience.

This thesis presents innovative methodologies to enable a truly portable and mobile VR experience, with lightweight VR glasses wirelessly connecting with edge/cloud computing devices that perform the rendering. There are two main challenges for this edge/cloud-based solution: (i) ultra-high bandwidth needed to transmit encoded video, and (ii) ultra-low latency needed to avoid user's dizziness feeling.

To address the challenging requirements of bandwidth and latency, this thesis presents three methodologies. Firstly, we have investigated and developed a novel hybrid-cast approach to save bandwidth in a multi-user streaming scenario. We identify and broadcast the common pixels shared by multiple users, while unicast the residual pixels for each user. We formulate the problem of minimizing the total bitrate needed to transmit the user views using hybrid-casting and present a common view extraction approach and a smart grouping algorithm to achieve our hybrid-cast approach. Secondly, we have explored 360-degree video and three Degrees of Freedom (3DoF) VR streaming scenario, and proposed a predictive adaptive streaming approach to reduce latency needed by pre-delivery. By streaming the predicted view with high predictive probability in relatively high quality according to bandwidth conditions and transmitted in advance, we aim to address both latency and constrained wireless bandwidth challenges. Thirdly, for six Degrees of Freedom (6DoF) VR content, we have developed predictive pre-rendering approach to reduce latency needed. By predicting both head and body motions, our approach can pre-render the predicted view in advance and thus save latency in 6DoF VR scenarios.

This thesis concludes with a summary of its contributions and open directions for future research.

At-home exercising strongly predicts physical therapy patient outcomes, underscoring the need for analyzing patient behaviors at-home via remote patient monitoring. Contemporary methods for remote patient monitoring rely on specialized sensors, i.e., Inertial Measurement Units, RGB-Depth cameras, motion capture systems, or stereo vision which are costly and not scalable to all physical therapy patients. Here, we observe a lack of literature using only a monocular RGB camera. In this paper, we demonstrate a skeletal feedback model for at-home exercises using only video acquired from a smartphone camera. We propose models for (i) Patient Performance Evaluation - which classifies the correctness of exercises, and (ii) Guidance - which identifies why the exercise went wrong so the patient can correct themselves. We use these models on our dataset of four common physical therapy exercises labeled by a physical therapist. Our results demonstrate the feasibility of using skeletal data from state-of-the-art 3D human pose estimation models for physical rehabilitation exercise evaluation and guidance. Thus, we enable remote patient monitoring and guidance from a single camera - making it highly cost-effective and scalable.

This study aims at using Road-side Unit (RSU)-assisted Vehicular Edge Computing (VEC) systems to support nowadays compute-intensive and delay-sensitive vehicular applications. Vehicles can offload these applications to the VEC server at the RSU to ease the burden on their limited onboard computing resources. Although VEC servers usually have higher computing capacity than vehicles, their computing resource, as well as the RSU's communication and energy resources, are not unlimited due to deployment constraints and operating costs. Our goal is to find the optimal resource allocation strategies for the RSU-assisted VEC systems to enable low-latency compute-intensive vehicular applications for vehicles.

In the first part of the study, we consider Solar-powered RSU-assisted VEC systems, where the RSUs are solely powered by solar energy. Firstly, we aim to minimize service disruption of the offloaded vehicular applications under the intermittent solar power supply. We propose a two-phase approach that jointly optimizes solar energy usage and storage, user association, and RSU’s computing and communication resource allocation for the involved RSUs and vehicles in the VEC system. Secondly, we further reduce the application’s execution delay by a framework that uses computing resources from both the VEC server and the vehicle's local computing (VLC) unit for application execution through task partitioning and offloading. Furthermore, the framework is able to adjust the VEC server’s platform configuration and balance between the vehicular application’s execution delay and its application-level performance, according to the available computing, communication, and solar energy resources of the Solar-powered RSU.

In the second part of the study, we focus on using the RSU-assisted VEC system to support an emerging advanced vehicular application, the multi-vehicle perception fusion. It is challenging to effectively allocate RSU’s computing and communication resources to support the multi-vehicle perception fusion application due to its complex and uncertain task composition natures. To minimize the end-to-end delay of the above application, we present a real-time mechanism that jointly determines the optimal RSU's computing and communication resource allocation, as well as task partitioning and scheduling strategies, which are adaptive to the dynamic task composition of the application.

Hypertension, or high blood pressure (BP), is one of the most prevalent chronic diseases in the world, affecting 30% of American adults and contributing to over 410,000 deaths per year. Although the effectiveness of lifestyle interventions on BP management has been proven in many studies, the degree of impact of each lifestyle factor on BP is unknown and may vary significantly between individuals. In this dissertation, we propose to use machine learning techniques to elucidate the complex relationships between BP and lifestyle factors at the level of the individual. Based on the continuous data collected from wearables of users, we aim to 1) provide a prediction of BP, which will give users a quick and reliable way to understand their health condition, and 2) provide personalized and actionable insight to users in order to control their BP by adjusting their lifestyle accordingly.Firstly, we extract necessary and interpretable features from lifestyle data. To utilize temporal information from the BP series, we propose to extract new features based on ARIMA to enhance the accuracy of BP prediction. We propose a machine learning method to explore the personalized relationship between BP and lifestyle factors collected by wearables. The proposed system provides BP prediction as well as lifestyle recommendations. Furthermore, since BP and health behavior data are collected and learned sequentially, the performance of prediction is prone to the existence of concept drifts and anomaly points. To solve this problem, we propose an Online Weighted-Resampling (OWR) technique to enhance RFFS in an online learning scenario. Thirdly, we propose a feature selection method using feature importance derived from Shapley Value, in order to remove redundant and irrelevant features and provide the personalized insight that may affect an individual’s BP. We evaluate and show that our proposed technique outperforms other popular machine learning methods in terms of prediction error. We also show the effectiveness of personalized recommendations using a randomized controlled trial. In the final section, earlier work in green communication, which utilize renewable energy to reduce grid energy consumption of base station (BSs), is presented. We propose to utilize data buffer of user equipment as well as energy storage at the BS to better adapt the BS resource allocation and hence its energy consumption to the dynamic nature of RE.

Anytime-anywhere connectivity offered by cellular networks and mobile devices

with multimedia capabilities have revolutionized important sectors of the society such

as health care, education, finance, e-commerce and entertainment. To cater to the

resulting explosive growth in mobile data traffic in an economically and environmentally

sustainable manner, it is critical to efficiently manage the spectral and energy/power

consumption of cellular networks. In this thesis, we identify the key challenges faced by

the cellular networks in efficiently managing energy/power consumption and propose

solutions to alleviate the same.

Rapid advances in processing capabilities of mobile devices and relatively slower

advances in battery capacity capabilities has created a huge gap between power required

for processing advanced multimedia applications and the available battery capacity. Data

and compute intensive mobile video is the leading multimedia application and leads to

quick drain in the mobile battery level. In the first part of the thesis, we address the

above challenge by developing battery aware mobile video download techniques that

increase the battery available time while maintaining the required user experience levels.

Extensive experiments have demonstrated the feasibility and efficacy of our approach.

Base stations are the dominant contributors to power consumption of cellular

networks. To ensure that quality of service requirements is always met, base stations are

over provisioned to handle maximum load and are always switched on. This is leads to

wasteful expenditure of electricity when load is less than maximum. To address this, we

develop techniques that adapt the coverage area of base stations depending on load to

reduce base station power consumption. Simulation experiments have demonstrated the

significant power savings is possible using the proposed techniques.

Multi-input, multi-output technologies which require multiple Radio Frequency

(RF) chains are being adopted to increase the data rates and coverage capabilities of

base stations. This implies that the already dominant contribution of RF chains to power

consumption of base stations will significantly increase. We conclude the thesis by

developing techniques that switch off RF chains depending on load to reduce base station

power consumption. Simulation experiments demonstrate the power savings possible

using proposed techniques compared to existing techniques.

Information explosion and the increasing use of Internet have fueled the growing popularity of personalization systems. Such systems can understand user interests to customize the information served to them, thereby addressing information overload. At the same time, personalization systems can also enable service and content providers to serve targeted advertisements and recommendations to users. In order to operate on the massive scales of the number of users and the amount of data available, personalization systems have a crucial requirement for automation and efficiency. In this dissertation, we identify key challenges faced by personalization systems and provide solutions to address them such that the above requirements can be achieved.

We first note that content classification is an important component of personalization systems. Approaches to train classification models typically depend on manual collection and labeling of training data, which makes personalization non-scalable. To address this, we develop a completely automated framework that can provide labeled training data for arbitrary set of categories. Experiments using online videos demonstrate the feasibility and effectiveness of our approach.

The second key challenge we address is the sparsity in annotations of popular online content such as Flickr images. Sparse or missing tags hamper the ability of personalization systems to recommend content or to infer the interests of users that access them. Towards this, we show how ontological tag trees can be constructed from corpus based statistics and semantic relationships between tags, to alleviate tag sparsity in a space efficient manner. Through evaluations, we demonstrate the effectiveness and efficiency of ontological tag trees as compared to existing methods.

Lastly, we focus on alleviating information overload in enterprise repositories. We design a file metadata based recommendation system that captures per user access patterns and user collaboration to recommend new files. In order to address scalability concerns of per user modeling, we propose optimizations that significantly reduce the time to serve recommendations to users. Experiments over actual enterprise data show that more than two orders of speed up is obtained as a result of the proposed methods.

To provide more optimal care at scale, health systems are changing the way in which healthcare is delivered. At the center of this changing landscape is a shift towards remote, continuous, and automated delivery of healthcare. This shift can lead to significant improvement in and scalability of at-home patient care for chronic diseases like hypertension and viral illnesses like COVID-19, while at the same time enabling significant savings in human and equipment resources. Wearable devices are an enabling technology making this shift in healthcare delivery possible due to the substantial amount of lifestyle and vitals data they can remotely collect. There is great opportunity for machine learning (ML) to assist in the remote and personalized delivery of care due to the large amount of data that is collected. In this dissertation, we present three applications of ML to enable personalized, remote health monitoring and care delivery. Chapter 1 presents a personalized deep learning approach to estimate blood pressure (BP) using the photoplethysmogram signal. Our approach enables continuous, noninvasive BP monitoring as compared to traditional methods which are either intermittent or invasive. To address the problem of limited personal data for individuals, we propose a transfer learning technique that achieves a mean absolute error of 3.52 and 2.20 mmHg for systolic and diastolic BP estimation, respectively. Chapter 2 describes a ML-based remote monitoring method to estimate patient recovery from COVID-19 symptoms using automatically collected wearable device data, instead of relying on manually collected symptom data. Our method achieves an F1-score of 0.88 when applying our Random Forest-based model personalization technique using weighted bootstrap aggregation. Chapter 3 presents the results of a single-arm nonrandomized trial which assessed the effectiveness of a fully digital, autonomous, and ML–based lifestyle coaching program on achieving BP control among adults with hypertension. 141 participants were monitored over 24 weeks and achieved an average systolic and diastolic BP decrease of 8.1 mmHg and 5.1 mmHg, respectively. Our research demonstrates the successful application of ML across various healthcare contexts. By harnessing wearable device data, we can facilitate more personalized and effective monitoring and interventions.