Research has suggested that social impressions of faces madeby Western and Eastern people have different underlying di-mensionalities. However, the individual level consistency, thegroup-level agreement of rater groups, and the interactionsbetween face ethnicity, rater ethnicity, and social impressiontraits remain largely unknown. In this paper, we perform alarge-scale data-driven cross-cultural study of facial impres-sions, and illustrate the idiosyncrasies and similarities behindCaucasian and Asian participants in their social impressions offaces from both ethnicity groups. Our study illustrates multi-ple interesting findings: (1) Asians rate faces lower on mostpositive traits, compared with Caucasian raters, and they havemore diverse opinions than Caucasians. (2) Caucasian faces re-ceive higher average ratings on social impression traits relatedto warmth due to the preponderance of smiles in Caucasianimages, but similar mean scores on traits related to capability,compared to Asian faces. (3) Caucasians and Asians disagreemost on capability related traits, especially on “responsible”and “successful.” Opinions on these two traits diverge moreon Asian than on Caucasian faces. Our findings provide newinsights on the nuances of cross-cultural differences in socialimpressions of faces.