Animal Minds in the Media: Learning outcomes for a critical-analysis assignment for students of comparative psychology

Students of comparative cognition must learn to read and evaluate scholarly writings such as journal articles and textbooks, and to think critically about information they hear from talks and lectures from experts in the field. They also must develop a healthy skepticism for popular-media portrayals of the mental and behavioral competencies of animals, whether those appear in serious formats such as documentaries and non-refereed popular science magazines or blogs, or even in media portrayals of animals that are intended purely for entertainment. Across a ten-year period, students in either a senior psychology course or a freshman honors seminar completed multiple assignments each semester called “Animal Minds in the Media” requiring identification and evaluation of popular media portrayals of the cognitive capabilities of animals, viewed through the lens of the comparative-psychology literature. The assignment was designed to motivate students to cultivate scientific skepticism and develop a “comparative psychologist’s way of seeing the world” by identifying implications or assumptions of popular-media treatment of animals and by bringing scientific literature to bear on the question of whether animals can actually think in the way implied by the commercial, comic, film, meme, or other media example.

The Looney Tunes character Wile E. Coyote planned elaborate traps to catch the Roadrunner. The Peanuts character Snoopy imagined himself a World War II flying ace. The Geico commercial's camel is happiest on Wednesday, or "hump day." Television's Lassie understood when Timmy was in trouble and how to recruit help for the little boy, and Frozen's Sven the reindeer understood that Kristof was in love and how best to help the lovelorn man. Media portrayals of animals-particularly portrayals like these and thousands of others that are intended for entertainment-are rife with anthropomorphism, creative license, and exaggerated competencies from enhanced perceptual abilities to full-blown human language. The point of these portrayals is not, of course, to claim that real coyotes can plan, that dogs can imagine or can offer help when it is needed, that camels understand days of the week (or can experience the emotion of happiness for that matter), and that reindeer understand the nature of love. The point of the assignment reviewed here is to ask about those animals, "But could they demonstrate those competencies?"

Teaching with Popular Media
Teachers have explored the use of popular media as instructional tools in a variety of psychology classes. For example, television commercials have been beneficial in psychology teaching, including for courses in research methods (Solomon, 1979), cognitive psychology (Gronlund & Lewandowsky, 1992), and for illustrating gender roles (Lloyd, 1999). Rahman and Zeglin (2014) reported favorable teaching outcomes from using comic books in abnormal-psychology instruction. Calcagno (2015) described the use of current news and popular media to provide concrete examples of abstract concepts in a psychology-of-women course. Gagnon (2016) was even more innovative in the use of popular media, developing a Facebook activity for teaching the history of psychology. Poonati and Amadio (2010) used television programs to help students understand operant conditioning. Many instructors (e.g., Alexander & Waxman, 2000;Fehim Kennedy et al., 2011;Hemenover et al., 1999;Paddock et al., 2001;Parlett, 2011;Petkari, 2017;Shepard & Brew, 2005;Smithikrai, 2016;Strelan, 2018;Wedding et al., 2017) have touted the benefits of popular movies to support psychology instruction, as summarized by this titular conclusion: "Russell Crowe is a better teacher than you" (Phillips, 2017).
Lanagan-Leitzel and Diller (2018) required students to submit weekly critical analyses of the psychological concepts that were on display in the TV series South Park. This use of popular media to teach critical thinking about psychological processes is closest to the purpose of the activity reported in the present paper. For the last decade, I had the privilege of developing and teaching a senior comparative-cognition course for psychology majors called "Animal Minds" as well as a version for freshman honors students of various majors. I have now taught the course as a third-year seminar at Covenant College. The course was organized around the scientific evidence (from journal articles, lectures, and a textbook) that might allow inferences about the mental lives of animals with respect to perception, attention, memory, learning, problem solving, numerical cognition, self-regulation and metacognition, language and communication, social cognition, and related constructs. The course is designed to teach discipline-specific critical thinking skills and to use writing assignments to stimulate and to assess students' improvement in the identification and evidence-based evaluation of arguments. Among the assignments that the students were required to complete in every section of this course is a critical-analysis exercise called "Animal Minds in the Media" (MiM).

Animal Minds in the Media
The syllabus description of the MiM assignment evolved a bit over years and was slightly different for the senior-psychology course compared to the freshman-honors sections. Notwithstanding, the general instructions were as follows: Provide a brief but critical analysis of how the mental activity or competency of animals is portrayed in movies, television, documentaries, comics, or other pop-culture examples. In each analysis, briefly describe one scene, or reproduce the image or URL. Write a one-page (200+words, not including any description of the scene) explanation of what the vignette assumes about animal minds, and evaluate whether you think these assumptions and arguments are accurate. Link your evaluation as much as possible to empirical evidence that you can find in the peer-reviewed literature, or we have discussed in class, or was in assigned readings. Avoid 'low hanging fruit' (e.g., 'This cartoon shows a cat talking, but cats can't talk') and find media messages that really make you think-and do some investigatingabout research on animal minds. See the example at the end of this syllabus.
Thus, the students' challenge was to analyze what assumptions or implications about animal behavior were in the media portrayal and then to evaluate whether actual animals could demonstrate the behavioral and/or cognitive competencies implied by the media example.
Each class was given an illustrative MiM analysis. The example used a Far Side cartoon, a humorous drawing that made no pretense of being scientifically grounded but that nonetheless raised implications about what dogs understand of human speech, the self-recognition implications of knowing their own name, and begged questions about whether dogs experience human-like emotions (e.g., shame, sadness) versus merely showing conditioned affective responding when scolded. There is comparative-psychology literature that bears on each of these topics-albeit maybe with species other than dogs-that allows the individuals to determine whether the scientific evidence indicates that canines or other animals can learn to comprehend speech or develop self-concepts or experience shame. Even where there are gaps in this literature, students can describe the research that should be done to provide evidence for evaluating the subtle claims of this and other media portrayals.
The goal for this assignment was four-fold. As mentioned above, the "Animal Minds" course was introduced as one of many "critical thinking through writing" senior-level seminars that were implemented by my institution, requiring students to conduct written critical analyses of popular media examples provided a perfect fit to this university-wide instructional quality-enhancement plan. Second, the assignment provided a novel way of encouraging students to explore the comparative-psychology literature: to find relevant sources, to evaluate the quality of empirical reports versus popular-media opinion, and to evaluate claims against the scientific evidence. Third, inference and anthropomorphism are early topics of discussion within the course, and classroom discussion was facilitated by student-located comics, movie scenes, television commercials, memes, and other media portrayals. Finally, a goal for the course was to stimulate students to see the world through the lens of experimental psychology generally and comparative cognition more specifically. That is, I hoped that students would come to see cute cat videos, the curious squirrel playing in the park, or some clever problem solved by an animal and think, "What (if anything) can be inferred about animal cognition from that anecdote?" Challenging students to think this way about the animated or fictitious animals in popular media seemed good practice for applying this same lens to the behavior of real animals.
Each semester, students were required to complete two or three different MiM analyses, with the first scheduled in the second or third week of the semester, and the other MiM analyses scheduled late in the term. Between these assignments, students received scheduled instruction (readings and lecture) about (a) the history of animal-minds research, (b) research methods in comparative psychology, (c) the challenges to inferring mental competencies from animal behavior, including the perils of anthropomorphism, the parsimony of lowerlevel explanations (e.g., chance, instinctive responses, conditioned behaviors), and (d) the state of the literature with respect to perception and attention, learning and memory, intelligence and problem solving, categorization and concept formation, numerical cognition and self-regulation, communication and language, and social cognition. The goal of this scheduling was to see whether improvement would be seen across the course of the semester, as students learned more of the comparative-cognition methods and literature and, of course, as they received feedback on their initial MiM submission. That feedback would include comments on the literature the students cited (or failed to cite), the understanding of the literature that they demonstrated, their consideration of evidence and counter-evidence, and grammar/stylistics. The third MiM, when assigned, was typically optional and for extra credit and will not be considered further in this paper.
Each MiM was graded by the author using a rubric that scored content development (i.e., whether the analysis was detailed and accurate), organization and logic, identification and use of evidence, position and balance (e.g., consideration of counter-arguments), and mechanics and grammar. Additionally, the rubric allowed for scoring of each MiM on three of the American Psychological Association's (APA's) undergraduateeducation learning outcomes associated with critical thinking, from the APA Guidelines for the Undergraduate Psychology Major (now in Version 2.0; see https://www.apa.org/ed/precollege/about/undergraduate-major; prior to the publication of Version 2.0, outcomes were scored according to the original list of outcomes but then retrofit to the current list; see Table 1).
For the present paper, learning-outcome and assignment grades will be examined for indications that the MiM activity was successful in stimulating critical thinking, in promoting appropriate identification and use of scholarly literature, and in helping students to learn about comparative cognition research. ☑ 2.2C Develop a comprehensive strategy for locating and using relevant scholarship (e.g., databases, credible journals) to address psychological questions ☑ 2.2D Evaluate psychology information based on the reliability, validity, and generalizability of sources ☑ 2.2E Interpret complex statistical findings and graphs in the context of their level of statistical significance, including the influence of effect size, and explain these findings using common language Goal 2.3 Engage in innovative and integrative thinking and problem solving ☑ 2.3A Describe problems operationally to study them empirically ☑ 2.3B Select and apply the optimal problem-solving strategy from multiple alternatives ☑ 2.3C Evaluate the effectiveness of selected problem-solving strategies * Goal 2 also included subgoals "2.4 Interpret, Design, and Conduct Psychological Research" and "2.5 Incorporate Sociocultural Factors in Scientific Inquiry" that were not directly relevant to this exercise.

Method and Results
Across the past decade, 12 sections of the "Animal Minds" course have been taught-6 at the senior psychology-major level, and 6 as a freshman seminar for honors students across majors. Thus, more than 350 individual MiM papers have been submitted and evaluated over the years. For the present analysis, 100 students (50 freshmen, 50 seniors) were selected at random to have both of their MiM papers included in this sample. (Students consented at the end of each course for their de-identified MiM papers to be used for purposes of quality control, scholarship of teaching and learning, and instruction.) Figure 1 displays the mean number of points (out of 50) awarded as a function of assignment number (1 = early in the semester, 2 = late in the semester) and course level (freshman, senior). Table 2 shows the rubric that was used for evaluating these 200 submissions. Ideas are arranged in a chaotic way, with no logical connection between them (i.e., within sentences and/or paragraphs).

pts
Ideas are arranged in an associative, digressive, or circular manner; the logical connections between ideas are consistently unclear.

pts
Ideas are arranged in a way that makes sense to the author, but is inappropriate for the purpose or audience of the assignment; the logical connections between ideas are frequently unclear. 6 pts Ideas are arranged with some consideration for the purpose and audience of the assignment; the logical connections between ideas are sometimes unclear. 8 pts Ideas are arranged in a manner appropriate to the purpose and audience of the assignment; the logical connections between ideas are almost always clear.

pts
Ideas are thoughtfully and effectively arranged in a manner appropriate to the purpose and audience of the assignment; the logical connections between ideas are consistently clear.

Use of Evidence 0 pts
Is unquestioning of his/her acceptance of unproven or poorly supported claims. 2 pts Is skeptical, but is unable to explain why or support claims with evidence. 4 pts Supports claims with evidence, but this evidence is may be inappropriate, inaccurate, or irrelevant. 6 pts Supports most claims with evidence that is appropriate, accurate, and relevant. 8 pts Consistently supports claims with appropriate, accurate, and relevant evidence. However, some important evidence is missing. 10pts Consistently and thoroughly supports claims with appropriate, accurate, and relevant evidence. Position and Balance 0 pts Does not take any position on the issue.

pts
Does not take a consistent position on the issue. 4 pts Takes a consistent position on the issue, but ignores relevant counterevidence and alternate points of view. 6 pts Takes a consistent position on the issue and attempts to address some relevant counterevidence and/or alternate points of view. 8 pts Takes a consistent position on the issue and addresses the relevant counterevidence and/or alternate points of view. However, claims and/or evidence could be presented more clearly and/or objectively. 10 pts Takes a clear and balanced position on the issue, thoroughly addressing relevant counterevidence and/or alternate points of view. Citation and Use of Sources 0 pts Has committed intentional plagiarism by copying and/or paraphrasing one or more sentences from a source and not using quotation marks or a citation to indicate the ideas are not his/her own. 1 pts Has committed unintentional plagiarism by inappropriately paraphrasing one or more sentences from a source (e.g., by changing some of the original words and/or the original word order), although a citation is provided.

pts
Quotes rather than paraphrases a cited source, or fails to provide citations or to discuss sources at all.

pts
Properly paraphrases all sources and provides citations for them, but has errors in citation mechanics.

pts
Properly paraphrases and cites all sources, but doesn't consistently or skillfully integrate the cited material with his/her own ideas.

pts
Properly paraphrases and cites sources, while consistently and skillfully integrating the cited material with his/her own ideas.
The effect of assignment number was statistically significant, F(1, 98) = 21.54, p < .001. Neither the class-level (freshmanhonors vs. senior-major) nor the interaction effects approached significance (ps > .10). The lack of difference between the senior psychology majors-overall, at the course, or at the end of the course-was surprising (and a bit disheartening!). Although the senior sections included some honors students and many talented psychology majors, the writing and critical thinking displayed by the freshman honors students showed an impressive ability and attention to the requirements of the assignment. As Figure 2 shows, points that were lost in MiM#1 were largely due to lack of knowledge or understanding of relevant literature against which to compare the media portrayal, and it was in this area that students of both levels showed the greatest improvement across the semester.

Figure 2
MiM#1 (Red Bars) and MiM#2 (Blue Bars) Average Points (Out of 10) for the Relevant Subgoals of the American Psychological Association's Undergraduate Guidelines (see Table 1) This pattern of results suggests that the students had improved in critical thinking skills across the semester, particularly with respect to identifying and evaluating evidence and using scientific rather than popular arguments to decide whether animals were demonstrating various cognitive competencies. It cannot be denied that grades may have improved for reasons other than growth in the students' capacities for critical and evidence-based thinking. Feedback received on the first MiM assignment included evaluations of spelling, grammar, and typographical errors that tended to decline across the two submissions, and students were much less likely to submit an analysis without citing any literature early in the semester. Additionally, the negative feedback on the first MiM report may have increased the students' motivation to work hard and earn a good grade (although it should be noted that MiM#1 grades were routinely curved, such that the students were not penalized for submitting a paper before we had actually begun reading or discussing the comparative-cognition literature; the data analyzed and reported in this paper are the unadjusted raw scores that students actually earned).
Of course, the primary limitation of these analyses is that the professor who was grading these assignments was the same person as the author of this paper and was aware of whether the media-portrayal papers were from early or late in the semester. It was obviously to the professor's advantage for students' performance, as a learning-outcomes measure, to improve across the term. Even if this were not the case, implicit biases may have resulted in improved scores for the second, relative to the first, MiM paper. To address these possibilities, two additional analyses were performed.
In the first, a sample of 40 papers (20 freshman, 20 senior; 10 from each class that were MiM#1 and 10 that were MiM#2 submissions) were selected at random and renamed arbitrarily by a student assistant, who also removed the students' names and any information that would identify whether the paper was an early-or a late-semester submission. Only the student assistant retained the record of which files corresponded to which condition. The author then re-graded each of these 40 papers using the original rubric, blind to the experimental conditions. The results of this re-scoring mimicked the findings shown in Figure 1. MiM#1 score was around 29.40 out of 50 for the freshmen, and 31.09 for the seniors. MiM#2 was approximately 40.00 for both groups.
Of course, it is possible that the author remembered which papers came from first-year or fourth-year course and whether the papers were early-or late-term submissions. (I wish my memory were this good!) As a second potential control, the sample of 40 papers was assigned to a current class of undergraduate students who are taking the "Animal Minds" class as a junior-level course. The students were about midway through the semester at the time that this assignment was made, and each had already submitted her or his own MiM#1 for the semester. The students were asked to evaluate 20 of the 40 papers, which the students could select arbitrarily. The rubric for the student evaluations is shown in Table 3.

Rubric for Undergraduate Raters' Blind Evaluations of Historical MiM Submissions
Rate each analysis on a 0 to 10 scale. Use the entire range, but here are some numbers to anchor your evaluations 10 pts Outstanding media example, animal-minds literature, critical analysis, organization, and position 7.5 pts Very good media example, animal-minds literature, critical analysis, organization, and position 5 pts Average media example, animal-minds literature, critical analysis, organization, and position 2.5 pts Fair media example, animal-minds literature, critical analysis, organization, and position 0 pts Very poor media example and animal-minds literature, critical analysis, organization, and position Because these MiM submission had been submitted years earlier at a different college by students that were unknown to the raters, there is little chance that the raters knew whether they were reading early-or late-semester submissions. Nevertheless, the ratings (averaged across the freshman and senior papers) had a mean of 5.2 for MiM#1 submissions, and 7.9 for MiM#2 submissions.
End-of-term feedback from students has revealed the MiM assignments to be among the more popular activities in the comparative-cognition class and easily the most popular of the writing-intensive options which, in some semesters, include a full-term paper or an APA-style research proposal on some animal-minds topic. Students reported that they enjoyed looking for media examples and thinking about the animal-minds implications of commercials, movies, comics, and the like. Two recurring suggestions to improve the assignment are (a) to distribute or post the best reports from classmates so that everyone can see them and (b) to schedule the first MiM assignment later in the semester so that the students know more about the topic. This second suggestion in and of itself provides another interesting endorsement of the activity as an instructional tool. The number of students who have suggested dropping the activity from the course entirely is approximately the same as the number who suggest implementing more of the analyses.

Summary
One goal for undergraduate training in psychology is to equip and motivate students to see the world as a psychologist does, that is, to show curiosity, skepticism, and critical thinking, while bringing scientific evidence to bear on opinions and beliefs. In this report, an instructional activity was described that has been used to stimulate students' curiosity, creativity, and critical acumen. Learning-outcome measures were reported to suggest that the activity has been effective in improving students' discipline-specific reasoning. It appears to be equally effective at the freshman or senior level or with majors and nonmajors alike. The "Animal Minds in the Media" assignment is an enjoyable exercise to discuss in a comparative-cognition course and has netted the instructor a large library of new comics, movie scenes, and so forth to use as examples or visual aids in the course! It is not unusual for a former student to email, even years after the class, with a media example that reminded the alumna/alumnus of the course and made her/him wonder, "Do animals really think in the way that this media portrayal is suggesting?"