Proceedings of the Annual Meeting of the Cognitive Science Society

We investigate people’s use of strategies for sampling data in an active learning task. In the spirit of resource-rational analysis, we argue that people may often use effective heuristics to guide sampling in lieu of more computationally expensive optimization strategies, but that when they encounter evidence that their heuristics are now ineffective they flexibly shift to new strategies. When the function family changed, participants quickly updated their beliefs about the likely function family on subsequent trials. By clustering participants’ sampling behaviour, we show that people can employ varied sampling strategies, shifting strategies more often when encountering unusual function families that are more adversarial to generic sampling strategies. Not all new strategies improved participants’ performance on a subsequent prediction task; nonetheless, people’s ability to dynamically shift their active learning behaviour may help them understand the abstract features of complex relationships.