Skip to main content
Open Access Publications from the University of California

Efficient learning through compositionality in a CNN-RNN model consisting of a bottom-up and a top-down pathway


Learning to write is characterized by bottom-up mimicking of characters and top-down writing from memory. We introduce a CNN-RNN model that implements both pathways: It can (i) directly write a letter by generating a motion trajectory given an image, (ii) first classify the character in the image and then determine its motion trajectory `from memory', or (iii) use a combination of both pathways. The results show that, in one-shot and few-shot learning, the model profits from different combinations of the pathways: The generation of different character variants works best when the top-down is supported by the bottom-up pathway. Refilling occluded images of efficiently learned characters works best when using the top-down pathway alone. Overall, the architecture implies that a weighted merge of bottom-up and top-down information into a latent, generative code fosters the development of compositional encodings, which can be reused in efficient learning tasks.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View