While neural networks are effective in classifying objects from highly structured data, their efficacy on unstructured point clouds has been more limited. In particular, their potential in classifying objects with subtle differences, such as body posture, has yet to be explored beyond simple gestures.Body language is an important type of communication as it enables people to ”speak” through their behaviors. However, the capability to understand body language varies under neu- rodivergent populations. From identifying common body poses, we could assist neurodivergent individuals in recognizing social cues from others or adjusting their own.
The aim of this thesis is to explore how deep learning can understand various poses in non-verbal communication and determine its potential in body language assistance. We test the ability of a deep neural network to classify poses from point cloud distributions recorded from a LiDAR sensor. Implementing dual-dimension blocks to the network improved performance by an average of 25% relative to the baseline provided by the original model, while adding hand-crafted features on top of that led to a 2-3% increase in accuracy. Alongside the overall improvements with dual-dimension annotations, the proposed network leads to improvements across non-background classes by an average of 2%. Based on our results and adjustments, we show how our network can identify body language based on the pose’s gesture and direction.