Some researchers claim that language acquisition is critically dependent on experiencing linguistic input in order of in-creasing complexity. We tested this hypothesis using a simple recurrent neural network (SRN) trained to predict wordsequences in CHILDES, a 5-million-word corpus of speech directed to children. First, we demonstrated that age-orderedCHILDES exhibits a gradual increase in linguistic complexity. Next, we compared the performance of two groups ofSRNs trained on CHILDES which had either been age-ordered or not. Specifically, we assessed learning of grammaticaland semantic structure and showed that training on age-ordered input facilitates learning of semantic, but not of sequentialstructure. Follow-up analyses suggest that higher noun-density in speech to younger children combined with weight en-trenchment could account for this effect. The persistent learning improvement is consistent with the neural commitmenthypothesis in the second language acquisition literature, which asserts that L1 representation reduces neural resourcesavailable for L2 learning. Similarly, exposure to noun-rich input first but not last (age-ordered CHILDES), may induce arepresentational advantage for lexical semantic acquisition.

How do children acquire semantic knowledge? In this work, we explore an old answer to this question: Semantic de-velopment is a hybrid of two distinct processes. The first process involves unsupervised learning of relations betweenobjects, providing a representation of objects that is useful for a wide range of possible goals. The second process involvesexplicitly learning to put objects and their relations into categories. Critically, this second process uses the representationsof the first process as its starting point. Here, we demonstrate this using a two-process model, where the first process is adistributional semantic model (e.g. HAL, Word2Vec, RNN), and the second process is a transformation of representationslearned during process 1 into a task-specific target space. This approach improves performance on multiple semantic tasks,compared to using the representations learned by process 1 directly. We believe this model demonstrates that a task- orgoal-oriented perspective of semantic cognition has promise for furthering our understanding of semantic development.

One proposal for how children acquire lexical categories is on the basis of their distributional signatures. Given that thelanguage children are exposed to gradually changes as they get older, it is possible that such changes impact the qualityof distributional information, and therefore the efficiency with which lexical categories are acquired. To test this idea, wecompiled a corpus of American-English child-directed speech and ordered it by increasing age of the target child. Next,we investigated the quality of distributional cues about lexical category membership in the first and second half of theage-ordered corpus. As predicted, we found that the quality of distributional information co-varies with age of the targetchild. Specifically, we found that distributional evidence for the noun category was of higher quality in speech to youngercompared to older children. In light of these findings, we recommend that distributional accounts of lexical categoryacquisition take into consideration language change during the first six years of development.

One proposal for how children acquire syntactic and semantic lexical categories is by inducing them from their distribu-tional signatures in speech. Because the language children are exposed to gradually increases in complexity as they getolder, it is possible that inducing lexical categories from initially simplified speech supports acquisition. We set out to testthis hypothesis using a simple recurrent neural network trained to predict 5 million words of child-directed speech from theAmerican-English portion of the CHILDES database. Evaluation of learned representations showed that models trainedin order in which children actually experience language performed better on a semantic, but not syntactic, categorizationtask. To understand why, we examined how the models encoded words during the earliest stages of training. Our resultsare relevant to important questions in language acquisition, such as the role of early experiences in organizing children’slinguistic representations.

We investigated neural networks’ ability to generalize during visual object recognition. In three experiments, we show that while basic multilayer neural networks easily learn to classify the objects on which they are trained, they show serious difficulties transferring that knowledge to novel items. However, our experiments also show that when the previously trained networks are then trained on the novel items, they learn to respond correctly to the novel items much faster than untrained networks. This shows that these networks are learning abstract representations that go beyond the simple items on which they were trained. We argue that this demonstrates that regarding abstract rule learning, the problem with neural networks is not their inability to learn abstractions, but their ability to apply that knowledge when classifying new objects.

An important aspect of language comprehension is learning and generalizing complex lexical relations. For instance, having learned that the phrase preserve cucumbers predicts vinegar and that preserve berries predicts dehydrator, one should be able to infer that the novel phrase preserve peppers is more compatible with vinegar, because pepper is more similar to cucumber. We studied the ability to perform such (compositional) generalization in distributional models trained on an artificial corpus with strict semantic regularities. We found that word-encoding models failed to learn the multi-way lexical dependencies. Recurrent neural networks learned those dependencies but struggled to generalize to novel combinations. Only mini GPT-2, a minified version of the Transformer GPT-2, succeeded in both learning and generalization. Because successful generalization in our tasks requires capturing the relationship between a phrase and a word, we argue that mini GPT-2 acquired hierarchical representations that approximate phrase structure. Our results show that, compared to older models, Transformers are architecturally advantaged to perform compositional generalization.

A critical part of language comprehension is inferring omitted but plausible information from linguistic descriptions of events. For instance, the verb phrase ‘preserve vegetable’ implies the instrument vinegar whereas ‘preserve fruit’ implies dehydrator. We studied the ability of distributional semantic models to perform this kind of semantic inference after being trained on an artificial corpus with strictly controlled constraints on which verb phrases occur with which instruments. Importantly, the ability to infer omitted but plausible instruments in our task requires compositional generalization. We found that contemporary neural network models fall short generalizing learned selectional constraints, and that a graph-based distributional semantic model trained on constituency-parsed data and equipped with a spreading-activation procedure for calculating semantic relatedness, achieves perfect performance. Our findings shed light on the mechanisms that give rise to compositional generalization, and using graphs to model semantic memory.

Importance

Magnetic resonance imaging markers of incipient cognitive decline among healthy elderly individuals have become important for both clarifying the biological underpinnings of dementia and clinically identifying healthy individuals at high risk of cognitive decline. Even though the role of hippocampal atrophy is well known in the later stages of decline, the ability of fornix-hippocampal markers to predict the earliest clinical deterioration is less clear.

Objectives

To examine the involvement of the hippocampus-fornix circuit in the very earliest stages of cognitive impairment and to determine whether the volumes of fornix white matter and hippocampal gray matter would be useful markers for understanding the onset of dementia and for clinical intervention.

Design

A longitudinal cohort of cognitively normal elderly participants received clinical evaluations with T1-weighted magnetic resonance imaging and diffusivity scans during repeated visits over an average of 4 years. Regression and Cox proportional hazards models were used to analyze the relationships between fornix and hippocampal measures and their predictive power for incidence and time of conversion from normal to impaired cognition.

Setting

A cohort of community-recruited elderly individuals at the Alzheimer Disease Center of the University of California, Davis.

Participants

A total of 102 cognitively normal elderly participants, with an average age of 73 years, recruited through community outreach using methods designed to enhance ethnic diversity.

Main outcomes and measures

Our preliminary hypothesis was that fornix white matter volume should be a significant predictor of cognitive decline among normal elderly individuals and that fornix measures would be associated with gray matter changes in the hippocampus.

Results

Fornix body volume and axial diffusivity were highly significant predictors (P = .02 and .005, respectively) of cognitive decline from normal cognition. Hippocampal volume was not significant as a predictor of decline but was significantly associated with fornix volume and diffusivity (P = .004).

Conclusions and relevance

This could be among the first studies establishing fornix degeneration as a predictor of incipient cognitive decline among healthy elderly individuals. Predictive fornix volume reductions might be explained at least in part by clinically silent hippocampus degeneration. The importance of this finding is that white matter tract measures may become promising candidate biomarkers for identifying incipient cognitive decline in a clinical setting, possibly more so than traditional gray matter measures.