When we encounter a person we rapidly form an impression. This impression is shaped by factors inherent to the person and event, including gender cues, skin colour, clothing, and context, along with pre-existing knowledge about the world. We present a PDP model of impression formation and person knowledge that has roots in the study of semantic cognition. We trained our model to learn about people of different races, genders, and occupations (e.g., black female doctor), along with context related to those occupations (e.g., hospital bed), and semantic attributes related to those occupations (e.g., intelligent). We demonstrate and explain the interesting finding that the model encodes knowledge about gender and race in early ‘visual’ hidden layers, despite no visual similarity being built-in, and no feedback connections from ‘semantic’ regions. This suggests a third mechanism through which the knowledge we hold about people may quite literally shape the way we see them.