UC San Diego

Facial attribute analysis plays a crucial role in various fields such as surveillance, entertainment, healthcare, and human-computer interaction. The advert of deep neural networks has sparked a growing interest in learning-based facial attribute analysis. This dissertation focuses on learning-based facial attribute analysis, encompassing facial attribute estimation and manipulation tasks. We focused on solving challenges including addressing data scarcity in supervised facial attribute estimation, handling facial attribute manipulation in high-resolution images, efficiently disentangling targeted attributes from others, and training a facial attribute manipulator with datasets from a small number of subjects.

The dissertation is structured around three key facial attribute categories: head orientations, gaze directions, and facial action units. The first part delves into improving appearance-based gaze estimation by considering person-dependent anatomical variations and accounting for ocular countering-rolling (OCR) responses, resulting in a more efficient and accurate method. The second part introduces ReDirTrans, a portable network designed for gaze redirection in high-resolution face images. By focusing on latent-to-latent translation, ReDirTrans enables precise gaze and head pose redirection while preserving other attributes, expanding its applicability beyond limited ranges of faces. The final part presents AUEditNet, a model for manipulating facial action unit intensities. This addresses challenges posed by data scarcity by effectively disentangling attributes and identity within a limited subject pool. AUEditNet demonstrates superior accuracy in editing AU intensities across 12 AUs, showcasing its potential for fine-grained facial attribute manipulation.

Overall, this dissertation contributes novel methodologies in learning-based facial attribute analysis, paving the way for enhanced performance and versatility across various real-world applications.