UC Irvine

Autonomous systems (AS) are beginning to play a significant role in modern society. Examples of AS include robotic systems that perceive and react to changes in their environment, such as aerial drones, ground/aquatic robots, and consumer autonomous vehicles (AVs), among others. Breakthroughs in deep learning and increased consumer interest in ubiquitous autonomy have fueled recent advances in perception, modeling, and control algorithms. However, these advances come with rising energy costs. Modern AS require large deep-learning (DL) models to perceive the environment and safely detect and avoid objects. The computational demands of these models significantly increase the hardware requirements of the complete system, such that modern AV systems can require several kilowatts of power to enable autonomy, reducing range and utility. These impacts affect most AS since AVs, ground robots, and drones are typically edge devices that operate in energy-constrained environments.

The need for large, complex DL models is primarily driven by the fact that, in the real world, challenging and unpredictable scenarios can occur and must be modeled appropriately by the AS. However, existing approaches are exceedingly cautious, preferring to execute a large, inefficient model even in typical scenarios just in case a difficult edge case arises. This approach to autonomy hinders both the utility of existing AS and the broad-scale adoption of AS. Instead, an AS should be able to understand the context of its surrounding environment and adapt its model to fit each situation.

This dissertation explores methods for developing context-aware AS to improve perception and prediction performance, reduce energy consumption, and enable adaptation to dynamic environments. Three different perspectives are studied:(i) scene-graph embedding approaches are proposed and evaluated for improved semantic understanding and state estimation; (ii) methods for implementing split ML models for dynamic, resource-efficient computation offloading from edge to cloud within real-time latency constraints are studied; and (iii) context-aware, dynamic ML perception models that jointly optimize performance and energy efficiency are studied from an algorithmic and system-wide optimization perspective. Overall, the results show that context-aware models achieve state-of-the-art performance across applications, and dynamic architectures conditioned on context can help resolve the energy-performance trade-off by enabling the joint optimization of these objectives.