UC Irvine

This dissertation addresses how contextualized expertise and task design can improve wisdom of the crowd estimates. The first two chapters apply the wisdom of the crowd to two related tasks that require spatial knowledge. The third chapter applies the wisdom of the crowd to a subset ranking task.

In Chapter 1, I investigate how framing effects impact the wisdom of the crowd. Participants selected tiles that either represented US states or African countries in two frames, present and absent. I constructed three wisdom of the crowd estimates: an unweighted average, a confidence-weighted average, and a wisdom of the crowd within estimate that combines an individual's responses across frames. I found that combining the estimates from the two frames resulted in an improved wisdom of the crowd estimate.

In Chapter 2, I build on the wisdom of the crowd application for a task that again requires spatial knowledge. Participants supplied a point estimate and a radius centered at that point estimate for where various US cities were located. Unweighted and radius-weighted wisdom of the crowd estimates were more accurate than most individuals, but the cognitive model-based wisdom of the crowd estimates tended to be even more accurate. I describe how using cognitive modeling that contextualizes expertise led to improved wisdom of the crowd estimates.

In Chapter 3, I present a new extension for the Thurstone model to partial ranking data. Ranking tasks have usually had participants rank all items, but I present two different types of partial ranking tasks where either an experimenter or a participant selects the items to be ranked. I demonstrate how the Thurstone model can be used to generate wisdom of the crowd estimates, and speculate how other partial ranking tasks can be developed to better elicit diverse estimates from the crowd.

In all, these chapters detail specific applications of the wisdom of the crowd effect that better contextualize expertise, elicit multiple meaningful estimates from the same individual, and improve diversity. These methods are used in conjunction with cognitive modeling to produce improved wisdom of the crowd estimates.