UC Irvine

Estimating and reconstructing human hand pose is a crucial task involved in many real world AI applications, such as human-computer interaction, augmented reality and virtual reality. However, hand pose estimation is challenging because the hand is highly articulated and dexterous, and hand pose estimation suffers severely from self-occlusion. To address the challenges of hand pose estimation from RGB images, several algorithms would be proposed in this thesis. In the first part, the task of 2D hand pose estimation from RBG images would be investigated. We introduce new techniques that combine traditional graphical probabilistic models with deep convolutional neural networks, and use these techniques to incorporate structural constraints of the hand to improve hand pose estimation. Apart from that, a novel graph neural network, spatial information aware GCN, would be proposed, which can efficiently extract spatial information from heatmaps of hand keypoints and propagate them through graph convolution. In the second part, the more challenging problem of 3D hand mesh reconstruction would be tackled. We will introduce an identity-aware hand mesh estimation network and a novel method to perform hand model calibration from RGB images. Extensive experiments have been conducted on multiple large-scale public datasets, demonstrating the state-of-the-art performance.