Proceedings of the Annual Meeting of the Cognitive Science Society

The Illusion of Explanatory Depth (IOED), i.e. the tendency to overestimate one's understanding, has been shown to increase when people search for explanations online. However, it remains unclear whether this effect extends to explanations obtained from AI chatbots. We examined whether chatbot use increases IOED and how it affects explanatory quality. In a between-subjects experiment, one group consulted ChatGPT on four topics, a second group read texts identical to the chatbot's responses without knowing their source, and a control group received no materials. Participants then rated their ability to explain each topic, produced written explanations, and evaluated their explanations. Results showed that the AI group exhibited the greatest overestimation of explanatory ability. In addition, both coder ratings and automated text analyses indicated lower explanation quality in the AI group compared to the non-AI group.