Proceedings of the Annual Meeting of the Cognitive Science Society

Effective problem solving requires both exploration and ex-ploitation. We analyze data from a group problem-solving taskto gain insight into how people use information from past expe-riences and from others to achieve explore-exploit trade-offs incomplex environments. The behavior we observe is consistentwith the use of simple, reinforcement-based heuristics. Partic-ipants increase exploration immediately after experiencing alow payoff, and decrease exploration immediately after expe-riencing a high or improved payoff. We suggest that whetheran outcome is perceived as “high” or “low” is a dynamic func-tion of the outcome information available to participants. Thedegree to which the distribution of observed information re-flects the true range of possible outcomes plays an importantrole in determining whether or not this heuristic is adaptive ina given environment.