Design is a complex cognitive task that pushes the limits of human information processing. How do expert designers handle this complexity? Professional and student architects solved a real-world diagram construction task that required satisfying multiple, sometimes conflicting, constraints to achieve an acceptable design. Professionals' initial designs were more consistent with task constraints and remained more consistent throughout problem solution. Students restructured their designs more often in their unsuccessful attempts to satisfy the multiple constraints imposed by the task. Analysis of subjects' verbal and action protocols suggests that one aspect of professionals' superior performance is their early recognition of the critical constraints on a design. Professionals handle these constraints before others to structure the remaining, more negotiable, constraints. By properly ordering constraints, professionals effectively minimize constraint conflicts. As conflict resolution has high processing costs, constraint prioritization may be one way that professionals cope with the complexity of design.

While recent studies have demonstrated various ways that transfer might be achieved in a domain, the measures used to assess transfer rarely stray from lime and error data. This paper examines transfer in the complex skill of computer programming in order to explore more flexible and sensitive methods of assessing transfer. In the experiment, subjects wrote both a P A S C A L and a LISP version of two programming problems. Although a simple accuracy measure provides evidence for knowledge transfer between the two programming languages, measures based on analyses of the task domain (i.e., partialcredit accuracy, strategy use) provide much stronger evidence. Curiously, these measures target different subjects as exhibiting transfer, suggesting that more than one type of knowledge may be available for transfer.

In this paper, we investigate students' use of the generate-and-test strategy to solve algebra word problems. This strategy involves first choosing an estimate for the answer and then checking whether the estimate satisfies the constraints of the problem. Based on verbal protocol data, we developed a production system model to simulate students' behavior when they apply this informal strategy. The model predicts problem features that should affect the difficulty of the problems. A large-scale experiment tested the predictions of the model. Verbal protocol data provided additional insights into how students use the generate-and-test strategy.

Verbal elaboration of a worked example has been shown to be helpful to learners before attempting to solve similar problems. This has been termed as the self-explanation effect. (Chi, Bassok, Lewis, Reimann & Glaser, 1989). This study examined how self-explanation changes before and after sequential problem solving rounds. We found that changes in self-explanation within an individual may affect individual performance across a series of problem solving episodes. Also, some participants appear to use the worked-out example as a self-generated feedback (SGF) mechanism to help with their problem solving rounds, while other participants do not. Locations or points in a worked-out example where self-explanation (elaboration) is most likely to occur for students with higher performance scores versus those with lower performance scores, is discussed. The implications of these differences for the design of a computational cognitive model are also addressed.

Conversations with computer agents can be used to measure skills that may be difficult to accomplish using tra-ditional multiple-choice assessments. In order to achieve natural conversations in this form of assessment, we are exploringissues related to how test-takers interact with computer agents, such as what dialogue moves lead to interpretable responses,the influence of “cognitive characteristics” of computer agents, how should the system adapt to test-taker responses, and howthese interactions impact test-taker emotions and affect. In this presentation we will discuss our current research addressingthese questions, illustrating important dimensions that are involved with designing a conversation space and how each designdecision can impact multiple factors within assessment contexts.