Much problem-solving research has investigated if and why ‘two heads are better than one’, but typically posits that if there is any process gain observed it is because of the exposure to the ideas provided by another person’s attempted solutions. This work fails to acknowledge or investigate what the interaction itself contributes to joint problem solving. Using an online version of the Alternative Uses Task, we compare situations in which people are passively exposed to what is said in a dialogue with situations in which people are actively engaged in the dialogue, thus varying the interactivity independently of the informational content that participants were exposed to. Interacting participants produce more turns overall, but they do not come up with more ideas. Interacting participants were also more likely to build on each other’s ideas and produce more complex ideas when a turn is linked to a previous idea; following leads to elaboration – but only if there is genuine interactivity. These results indicate that conversational mechanisms promote the exploration of a problem space and that merely counting the number of ideas produced would miss the importance of the interaction itself.