Proceedings of the Annual Meeting of the Cognitive Science Society

Assessing the quality of a learner’s solution for a given task isan essential step in analyzing a learner’s performance. For awell-defined sequential problem, correctness and optimality ofthe solution as well as its length provide first simple and rea-sonable metrics. However, this ignores the fact that there areconceptually different errors that humans make when solving aproblem. This work proposes a rule-based system of error cat-egories which is able to classify conceptually different errorswith respect to their (assumed) motive. The principles the cat-egories are based on are valid for most well-defined sequentialproblems and can hence serve as a valuable tool in the analy-sis of human solutions for such a problem. In this work, theerror category system is adapted to the game Rush Hour. Weuse the category system as a tool for a detailed analysis of 115human solutions of a Rush Hour game. We found that the mostcommon error type is based on a simple solving heuristic, butmainly occurs in the first half of the solution process. Other er-ror types whose occurrence is numerically less dominant, arestill found in the majority of the solutions. However, they oc-cur in very specific game situations. As a first generalizationapproach of the category system, its application on a furtherdataset containing 56 different Rush Hour tasks and more than31, 000 human solutions yield promising results.