Skip to main content
eScholarship
Open Access Publications from the University of California

UCLA Criminal Justice Law Review

UCLA Criminal Justice Law Review bannerUCLA

The (Im)Proficiency of Firearms Examiners

Abstract

For decades, the nation’s largest provider of forensic proficiency tests, Collaborative Testing Services, structured its firearms examination exercises around bullet and cartridge case comparisons so simplistic as to nearly guarantee vanishingly low misidentification rates. Practitioners, in turn, enthusiastically relied on these results during increasingly vigorous admissibility challenges to defend their alleged ability—using little more than their training, experience, and subjective judgment—to divine the precise gun used in a crime. By and large, they succeeded. Although outside scholars (lawyers, statisticians, human factors experts, and more) have remained constant in their criticisms of the field’s methods and the design of its efforts to measure accuracy (in the form of proficiency testing and beyond), untroubled admissibility and untroubled qualification of firearms examiners as “experts” have remained the norm. In 2023, however, CTS (albeit unintentionally) bucked the trend of simplistic testing, releasing an exercise, CTS 23–5262, that forced participants to confront damaged and poorly marked bullets. The results should trouble even those unfamiliar with firearms examination and the debates about its validity. Although the questioned bullets provided to participants were not fired by the same gun as the exemplar bullets in their packets, 22% of test takers failed by “matching” at least one questioned item to the exemplars, and participants on the whole committed misidentifications 37.4% of the time they offered definitive conclusions about the source of the “challenging” bullets in the proficiency test. To highlight the importance of these results, and to emphasize the extent they depart from past proficiency tests, this Article both analyzes CTS bullet exercises dating back to 2006 and dives deeply into CTS 23–5262 itself. Examiners’ performance failures on the latter, this Article ultimately argues, (1) contradict claims that analysts will rarely err when assessing the source of fired ammunition; (2) raise questions about the validity of comparisons performed on damaged bullets; and (3) force litigants and the courts to grapple with the significance of test error when qualifying individual examiners as experts. Indeed, given how grossly CTS 23–5262 runs counter to the claims made by defenders of firearms examination (and the judges who have parroted them), it may be that the test can (or at least should) finally push the field over the edge and out of criminal courts.