Proceedings of the Annual Meeting of the Cognitive Science Society

Large language models are increasingly being used for education and science communication by automatically generating explanations of scientific concepts. However, prior research has found that the analogies produced by LLMs lack human-like psycholinguistic properties important for readability. In this work, we propose cognitive conflict reinforcement learning (CCRL) to improve the psycholinguistic properties of analogies generated by LLMs. Specifically, we create cognitive conflict between the original LLM and a cloned LLM during reinforcement learning. This helps address the cognitive rigidity problem in LLMs. Experimental results demonstrate that our approach significantly outperforms existing RL algorithms and human performance in improving various readability metrics of generated analogies.