Proceedings of the Annual Meeting of the Cognitive Science Society

Averaging the estimates of a number of individuals has beenshown to produce an estimate that is generally more accuratethan those of the individuals themselves. Similarly, averagingresponses from a single individual can also lead to a moreaccurate answer. How can we best combine estimates withinand between individuals to create an accurate group estimate?We report empirical results from a general knowledge rank-ordering experiment and demonstrate that individuals thatprovide more consistent answers across repeated elicitationsare also more accurate. We develop a consistency weightingheuristic and show that repeated elicitations within anindividual can be used to improve group accuracy. We alsodevelop a Thurstonian cognitive model which assumes adirect link between the process that explains the accuracy ofan individual and response consistency and show how themodel can infer accurate group answers.