UC San Diego

Facial impressions play a crucial role in real life, affecting decisions from dating choices to electoral outcomes. Globalization has made it increasingly important to understand how impressions are formed across different cultures. In this thesis, we conduct a large scale data-driven study on cross-culture social impressions of faces. We start by collecting impression ratings of Chinese Asians and U.S. Caucasians on Chinese and Caucasian faces, for 18 social traits related to warmth, attractive-youth, competence and sexual dimorphism. By analyzing the collected data, we observe some widely-agreed upon cultural universals in how high-level facial features relate to impressions. On the other hand, we also find the following cultural differences: (a) Asians give overall lower positive impression ratings to faces compared to Caucasians. (b) raters from both cultures agree more on warmth-related traits, but (c) less on competence-related traits. Furthermore, we introduce a simple and interpretable model, CultureNet, to predict the social impression ratings of a given facial image on specific social traits. The model leverages on the correlation structure of social impression traits and uses them as an attention map that activates different areas of image features. We find CultureNet is able to outperform alternative models with a small number of parameters. Moreover, the trait embedding and activation visualization generated by CultureNet allow us to interpret how the predictions are made. Our work provides a novel method of understanding what features drive facial impressions and allows us to compare perceptions across different cultures.