Proceedings of the Annual Meeting of the Cognitive Science Society

Learning to write is characterized by bottom-up mimicking of characters and top-down writing from memory. We introduce a CNN-RNN model that implements both pathways: It can (i) directly write a letter by generating a motion trajectory given an image, (ii) first classify the character in the image and then determine its motion trajectory `from memory', or (iii) use a combination of both pathways. The results show that, in one-shot and few-shot learning, the model profits from different combinations of the pathways: The generation of different character variants works best when the top-down is supported by the bottom-up pathway. Refilling occluded images of efficiently learned characters works best when using the top-down pathway alone. Overall, the architecture implies that a weighted merge of bottom-up and top-down information into a latent, generative code fosters the development of compositional encodings, which can be reused in efficient learning tasks.