UC Riverside

Most human recognition methods rely on unique bio-metric features like facial structure, fingerprints, iris, voice, hand, or gait. However, human body shape also offers a distinctive metric for identification and is very important for various applications, including crime prevention, forensic identification, and security monitoring, especially when the face of a person cannot captured by camera. Recently, with the raising of 3D objects in today's technology landscape, especially in virtual reality (VR) and augmented reality (AR), people have created and enhanced many methods to regress accurate 3D human bodies from one or many RGB images. These methods are really helpful not only in the field of creating 3D objects in AR or VR but also in human recognition. Many human recognition systems recently integrated these 3D object regression methods in their neural network to make their deep learning models understand more about humans in 3D. This raises the question of, whether these 3D objects are accurate enough to recognize humans. Almost no research has tried to recognize humans on 3D objects. Our research aims to figure out whether these 3D human bodies regressed from 2D images from the state-of-the-art (SOTA) method are good enough to recognize humans. We used SHAPY, a SOTA model for generating a 3D human body mesh from an RGB image. While deep learning has recently shown notable results in processing directly point cloud to do some tasks related to 3D objects like 3D object classification, or segmentation, it is very hard to explain the results of deep learning methods. Therefore we are using Local Surface Patch (LSP) which is a geometric way to extract shape features from 3D meshes. Local Surface Patch makes it very easy to verify if recognition in 3D mesh from SHAPY works, and if it doesn’t work, we can explain why. We have done a lot of experiments to show that 3D meshes created from the state-of-the-art method are currently not good enough for human body recognition. We also implement and improve LSP methods for 3D objects and show that it is robust to capture shape features from 3D objects. We also propose a solution to alleviate the effect of high variance 3D body shape from SHAPY and improve the recognition results.