Artificial Neural Networks can be used to build a general theory of intelligent systems, connecting the computational, algorithmic and implementational levels. I analyze the generalization of learning in simple but challenging problems as a way to build the theory. I report simulations of learning and generalizing sameness, using Simple Recurrent Networks (SRN), Long-Short Term Memories (LSTM) and Transformers. We show that even when minimal requirements to implement sameness in SRNs are met, and a SRN network that can compute sameness theoretically exists, we failed to obtain it by training with backpropagation using all the possible input pairs. LSTMs come close to learn sameness, but the best networks require an inordinate amount of examples and the enrichment of the sample with positive examples. The same happens with Transformers. A similar task applied to ChatGPT revealed related problems. We discuss what this implies for Cognitive and Neural Sciences.

Although Artificial Neural Networks were born as Neurocognitive models, the architectures used nowadays in AI are not conceived as models of the brain. In the last few years, Deep networks have been fitted to brain activity to use them as neural models, but their use as cognitive models is less prevalent. Here we use a transformer model complemented with a simplified visual input to model reading acquisition. First, we trained the network to recognize the speech input. After that, we use letter sounds and letter visual representations to train the network to output the correct letters. We apply this model to our empirical previous results, comparing learning in a transparent (Spanish) and an opaque (French) orthography as in transparent orthographies, phonological awareness is much less important than in opaque orthographies as a predictor of reading. We show that the difficulty of training correlates with opaqueness, and interpret the results.

Word embeddings trained on large scale text corpora are central to modern natural language processing and are also important as cognitive models and tools in psycholinguistic research. An important alternative to these text-based models are embeddings derived from word association norms. Recently, these association-based embeddings have been shown to outperform text-based word embeddings of comparable complexity (such as GloVE, word2Vec & fastText) in semantic similarity rating tasks. Here we evaluate English and Rioplatense Spanish association-based embeddings derived from the Small World of Words (SWOW) project on the Google Analogy set and the Bigger Analogy Test Set. We also developed a small analogy set that focuses on semantic relationships, such as event knowledge and category-exemplar relationships such as prototypicality. SWOW-derived word embeddings perform similarly as traditional text-based word embeddings in semantic analogies, and outform them in some categories. These results illustrate relevant similarities and differences between text-based and word association-derived embeddings.

Word embeddings trained on large scale text corpora are central to modern natural language processing and are also important as cognitive models and tools in psycholinguistic research (Pennington et al., 2014). An important alternative to these text-based models are embeddings derived from word association norms (De Deyne et al., 2019). Recently, these association-based embeddings have been shown to outperform text-based word embeddings of comparable complexity (such as GloVE, word2Vec & fastText) in semantic similarity rating tasks (Cabana et al., 2023; Richie & Bhatia, 2021). Here we evaluate English and Rioplatense Spanish association-based embeddings derived from the Small World of Words (SWOW) project on the Google Analogy set and the Bigger Analogy Test Set (Gladkova et al., 2016). We also developed a small analogy set that focuses on semantic relationships, such as event knowledge and category-exemplar relationships such as prototypicality. SWOW-derived word embeddings perform similarly as traditional text-based word embeddings in semantic analogies, and outform them in some categories. These results illustrate relevant similarities and differences between text-based and word association-derived embeddings. References Cabana, Á., Zugarramurdi, C., Valle-Lisboa, J. C., & De Deyne, S. (2023). The “Small World of Words” free association norms for Rioplatense Spanish. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02070-z De Deyne, S., Navarro, D. J., Perfors, A., Brysbaert, M., & Storms, G. (2019). The “Small World of Words” English word association norms for over 12,000 cue words. Behavior Research Methods, 51(3), 987–1006. https://doi.org/10.3758/s13428-018-1115-7 Gladkova, A., Drozd, A., & Matsuoka, S. (2016). Analogy-based detection of morphological and semantic relations with word embeddings: What works and what doesn't. Proceedings of the NAACL Student Research Workshop, 8–15. https://doi.org/10.18653/v1/N16-2002 Richie, R., & Bhatia, S. (2021). Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison—Richie—2021—Cognitive Science—Wiley Online Library. Cognitive Science, e13030. https://doi.org/10.1111/cogs.13030