Proceedings of the Annual Meeting of the Cognitive Science Society

Biological agents refine behavioral strategies through offline consolidation without real time environmental interaction. Cognitive science suggests this capability stems from constructive memory, where the brain actively reconstructs the past guided by current goals rather than mechanically replaying it. Drawing on this, we propose the Value Guided Action Reconstruction model to operationalize this mechanism for offline reinforcement learning. Specifically, we model policy learning as a value guided generative process. By injecting value signals into a variational autoencoder latent space, we simulate mental rehearsal prioritizing high value behaviors. To mitigate memory distortion risks, we introduce a metacognitive confidence monitoring mechanism. This module dynamically regulates value guidance intensity by evaluating the discrepancy between reconstructed actions and original traces, thereby balancing optimization with fidelity. Empirical evaluations on MuJoCo and Maze2d tasks demonstrate that our model enhances robustness, highlighting the computational advantages of constructive memory in facilitating offline generalization.