Proceedings of the Annual Meeting of the Cognitive Science Society

More and more people use large language models (LLMs) as sources of information. However, LLMs are prone to hallucinations, meaning they can produce plausible but false information. In a pre-registered laboratory experiment, we analyzed the effects of hallucination warnings (between-subjects: with vs. without) and source credibility (within-subjects: high-credible vs. unknown vs. low-credible) on content learning and source memory. N = 97 learners first received pieces of information accompanied by a source label from a simulated LLM-based chatbot, before completing learning and source memory tests. Source memory was analyzed with multinomial processing tree models. Content learning was not affected by warnings and source labels. With hallucination warnings, high- and low-credible sources were remembered better than unknown sources. However, without a warning, only low-credible sources were remembered better than unknown sources. Disclosing the source (credibility) seems promising for source memory, but especially high-credible sources may require additional highlighting in contexts without warnings.