A pressing issue in both psychology and agent-modeling com-munities is the inability to account for the wide variance in hu-man variability and individual differences. Added to this is thefurther complexity of changing goals and social meaning in adynamic, sequential interaction. While prior work on artificialagent design has prominently addressed physical cues and non-verbal behavior, there is a lack of emphasis on (1) examiningcues in combination, and (2) assessing judgments of social sit-uational meaning. In the current work, we present an ontologyof physical behavior (Social Kinematics) that accounts for thecombinatorial effects of multiple cues, as well as the changingsocial meaning associated with these different combinations ofcues. Here, we assess individuals social situational judgmentsof multiple combinations of ambiguously-defined virtual agentanimations. Ultimately, this paper provides a potentially usefulframework that has relevance for researchers in social robotics,agent modeling, and cognitive science.