Skip to main content
eScholarship
Open Access Publications from the University of California

Scholarly Works (8 results)

  • Thesis
  • Peer Reviewed
UC San Diego Electronic Theses and Dissertations (2020)

At first sight of a new person, people quickly form impressions on how attractive, trustworthy, and warm the person looks. Despite the dubious accuracy of these first impressions, people rely on them to navigate social interactions regarding interpersonal relationships, political voting decisions, and financial choices. As we progress into a digital era with frequent social media usage and cultural exchanges, it is crucial to understand the universal principles and cultural idiosyncrasies of impression formation from face images.

In this thesis, we examine social impressions of faces through the lens of computational modeling, psychological experiments, and cultural comparisons. First, we develop a model that automatically predicts human social impression judgments using neural networks. Building on the predictive model, second, we build a generative model that can change faces holistically to augment or decrease specific characteristics, such as attractiveness and aggressiveness of a face. Third, we examine how specific physical attributes, such as hair color, affect impressions of faces, using a GAN model, and psychological experiments. Finally, we conduct a large-scale cross-cultural study, using 18 traits related to approachability, youthful-attractiveness, and competence evaluation, with Caucasian and Asian participants rating Caucasian and Asian faces. We investigate the mediating factors behind impression formation and estimate how high-level facial features such as age, eyeglasses, and smiles are judged similarly and differently by people from two cultures.

Overall, our work provides a computational framework to predict and modify faces, which is practically useful in real-life scenarios regarding optimal self-image presentation. Our model lays the foundation for the quantitative study of first impressions. Our psychological and cross-cultural studies reveal the universality and culture-specific judgments mediated by high-level features that affect first impressions. These findings motivate future research in social psychology to understand the deeper cultural roots behind the differences in first impression formation and be aware of the potential bias toward certain social groups.

    Cover page: Social Impressions of Faces: Computational Modeling and Cultural Comparisons
    • Article
    • Peer Reviewed
    Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 41 (2019)
    At first glance, humans extract social judgments from faces, in-cluding how trustworthy, attractive, and aggressive they look.These impressions have profound social, economic, and polit-ical consequences, as they subconsciously influence decisionslike voting and criminal sentencing. Therefore, understand-ing human perception of these judgments is important for thesocial sciences. In this work, we present a modifying autoen-coder (ModifAE, pronounced “modify”) that can model andalter these facial impressions. We assemble a face impressiondataset large enough for training a generative model by ap-plying a state-of-the-art (SOTA) impression predictor to facesfrom CelebA. Then, we apply ModifAE to learn generalizablemodifications of these continuous-valued traits in faces (e.g.,make a face look slightly more intelligent or much less aggres-sive). ModifAE can modify face images to create controlledsocial science experimental datasets, and it can reveal datasetbiases by creating direct visualizations of what makes a facesalient in social dimensions. The ModifAE architecture is alsosmaller and faster than SOTA image-to-image translation mod-els, while outperforming SOTA in quantitative evaluations.
      Cover page: Modifying social dimensions of human faces with ModifAE
      • Article
      • Peer Reviewed
      Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 38 (2016)
      Facial attractiveness has long been a topic of interest for cognitive scientists. Early psychological research has foundthat averageness, distinctiveness and familiarity of a face can influence facial attractiveness. However, faces also convey richsocial information. How various social features are related to facial attractiveness hasn’t been systematically studied before.We investigate facial attractiveness in the context of social feature evaluation and find that social attributes like appearinginteresting and sociable contribute to facial attractiveness whereas appearing boring, and humble are negatively correlated withattractiveness. We further compare social features of faces with the physical configuration of faces and we are able to usegeometric features to predict facial attractiveness. We further study the individual differences on attractiveness perception andfind out that. Our study illustrates that social attributes and pixel information can go hand in hand to facilitate attractivenessprediction.
        Cover page: Understanding human facial attractiveness from multiple views
        • Article
        • Peer Reviewed
        Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 38 (2016)
        Two experimental protocols, pairwise rating and triplet rank-ing, have been commonly used for eliciting perceptual similar-ity judgments for faces and other objects. However, there hasbeen little systematic comparison of the two methods. Pairwiserating has the advantage of greater precision, but triplet rank-ing is potentially a cognitive less taxing task, thus resulting inless noisy responses. Here, we introduce several information-theoretic measures of how useful responses from the two pro-tocols are for the purpose of response prediction and parame-ter estimation. Using face similarity data collected on AmazonMechanical Turk, we demonstrate that triplet ranking is signif-icantly better for extracting subject-specific preferences, whilethe two are comparable when pooling across subjects. Whilethe specific conclusions should be interpreted cautiously, dueto the particularly simple Bayesian model for response gener-ation utilized here, the work provides a information-theoreticframework for quantifying how repetitions within and acrosssubjects can help to combat noise in human responses, as wellas giving some insight into the nature of similarity representa-tion and response noise in humans. More generally, this workdemonstrates that substantial noise and inconsistency corruptsimilarity judgments, both within- and across-subjects, withconsequent implications for experimental design and data in-terpretation.
          Cover page: Extracting Human Face Similarity Judgments: Pairs or Triplets?
          • Article
          • Peer Reviewed
          Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 39 (2017)
          Humans make complex inferences on faces, ranging from ob-jective properties (gender, ethnicity, expression, age, identity,etc) to subjective judgments (facial attractiveness, trustworthi-ness, sociability, friendliness, etc). While the objective as-pects of face perception have been extensively studied, rela-tively fewer computational models have been developed forthe social impressions of faces. Bridging this gap, we de-velop a method to predict human impressions of faces in 40subjective social dimensions, using deep representations fromstate-of-the-art neural networks. We find that model perfor-mance grows as the human consensus on a face trait increases,and that model predictions outperform human groups in cor-relation with human averages. This illustrates the learnabilityof subjective social perception of faces, especially when thereis high human consensus. Our system can be used to decidewhich photographs from a personal collection will make thebest impression. The results are significant for the field of so-cial robotics, demonstrating that robots can learn the subjectivejudgments defining the underlying fabric of human interaction.
            Cover page: Learning to See People Like People: Predicting Social Impressions of Faces
            • Article
            • Peer Reviewed
            Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 42 (2020)
            Research has suggested that social impressions of faces madeby Western and Eastern people have different underlying di-mensionalities. However, the individual level consistency, thegroup-level agreement of rater groups, and the interactionsbetween face ethnicity, rater ethnicity, and social impressiontraits remain largely unknown. In this paper, we perform alarge-scale data-driven cross-cultural study of facial impres-sions, and illustrate the idiosyncrasies and similarities behindCaucasian and Asian participants in their social impressions offaces from both ethnicity groups. Our study illustrates multi-ple interesting findings: (1) Asians rate faces lower on mostpositive traits, compared with Caucasian raters, and they havemore diverse opinions than Caucasians. (2) Caucasian faces re-ceive higher average ratings on social impression traits relatedto warmth due to the preponderance of smiles in Caucasianimages, but similar mean scores on traits related to capability,compared to Asian faces. (3) Caucasians and Asians disagreemost on capability related traits, especially on “responsible”and “successful.” Opinions on these two traits diverge moreon Asian than on Caucasian faces. Our findings provide newinsights on the nuances of cross-cultural differences in socialimpressions of faces.
              Cover page: Do you see what I see?A Cross-cultural Comparison of Social Impressions of Faces
              • Article
              • Peer Reviewed
              Proceedings of the Annual Meeting of the Cognitive Science Society, Volume 42 (2020)
              How does hair color affect people’s first impression of a face? Equipped with state-of-the-art Generative AdversarialNetwork (GAN) models, we are able to re-investigate the questions with strictly and precisely controlled image stimuli.By creating triplets of the same face image with different hair colors, we examine how black/brown/blond hair colorsaffect perception of attractiveness, trustworthiness and intelligence. Our study finds that if the original hair color is dark,the optimal choice in most cases is to stay in dark colors. If your original hair color is blond, changing into brown will,in general, make you look more intelligent, sometimes at the cost of attractiveness. The specific best color choice variesa lot more for people with blond hair. Furthermore, we train a neural network model that predicts people’s impressionson faces in different trait dimensions accurately. This study could provide guidance to people regarding their image andimpression control.
                Cover page: To Dye or Not to Dye : The Effect of Hair Color on First Impressions
                • Article
                • Peer Reviewed
                UC San Francisco Previously Published Works (2018)

                Objective

                A pilot study suggested that combination therapy with low-dose anti-thymocyte globulin (ATG) and pegylated granulocyte colony-stimulating factor (GCSF) preserves C-peptide in established type 1 diabetes (T1D) (duration 4 months to 2 years). We hypothesized that 1) low-dose ATG/GCSF or 2) low-dose ATG alone would slow the decline of β-cell function in patients with new-onset T1D (duration <100 days).

                Research design and methods

                A three-arm, randomized, double-masked, placebo-controlled trial was performed by the Type 1 Diabetes TrialNet Study Group in 89 subjects: 29 subjects randomized to ATG (2.5 mg/kg intravenously) followed by pegylated GCSF (6 mg subcutaneously every 2 weeks for 6 doses), 29 to ATG alone (2.5 mg/kg), and 31 to placebo. The primary end point was mean area under the curve (AUC) C-peptide during a 2-h mixed-meal tolerance test 1 year after initiation of therapy. Significance was defined as one-sided P value < 0.025.

                Results

                The 1-year mean AUC C-peptide was significantly higher in subjects treated with ATG (0.646 nmol/L) versus placebo (0.406 nmol/L) (P = 0.0003) but not in those treated with ATG/GCSF (0.528 nmol/L) versus placebo (P = 0.031). HbA1c was significantly reduced at 1 year in subjects treated with ATG and ATG/GCSF, P = 0.002 and 0.011, respectively.

                Conclusions

                Low-dose ATG slowed decline of C-peptide and reduced HbA1c in new-onset T1D. Addition of GCSF did not enhance C-peptide preservation afforded by low-dose ATG. Future studies should be considered to determine whether low-dose ATG alone or in combination with other agents may prevent or delay the onset of the disease.
                  Cover page: Low-Dose Anti-Thymocyte Globulin (ATG) Preserves β-Cell Function and Improves HbA1c in New-Onset Type 1 Diabetes
                  Top