Most work in learnability theory assumes that both the environment (the data to be learned) and the learning mechanism are static. In the case of children, however, this is an unrealistic assumption. First-language learning occurs, for example, at precisely that point in time when children undergo significant developmental changes. In this paper I describe the results of simulations in which network models are unable to learn a complex grammar when both the network and the input remain unchanging. However, when either the input is presented incrementally, or—more realistically—the network begins with limited memory that gradually increases, the network is able to learn the grammar. Seen in this light, the early limitations in a learner may play both a positive and critical role, and make it possible to master a body of knowledge which could not be learned in the mature system.

Recent descriptions of connectionist models have argued that connectionist representations are unstructured, atomic, and bounded(e.g., Fodor & Pylyshyn, 1988). This paper describes results with recurrent networks anddistributed representations which contest these claims. Simulation results are described which demonstrate that connectionist networks are able to learn representations which are richly structured and open-ended. These representations make use both of the high dimensional space described by hidden unit patterns, as well as trajectories through this space in time, and posses a rich structure which reflects regularities in the input.Specific proposals are advanced which address the type/token distinction, the representation of hierarchical categories in language, and there presentation of grammatical structure.

Recent PDP models have been shown to have great promise in contributing to the understanding of the mechanisms which subserve language ixt)cessing. In this paper we address the specific question of h o w multiply embedded sentences might be processed. It has been shown experimentally that comprehension of center-embedded structures is poor relative to right-branching structures. It also has been demonstrated that this effect can be attenuated, such that the presence of semantically constrained lexical items in center-embedded sentences improves processing performance. This raises two questions: (1) What is it about the processing mechanism that makes center-embedded sentences relatively difficult? (2) H o w are the effects of semantic bias accounted for? Following an approach outlined in Elman (1990, 1991), w e train a simple recurrent network in a prediction task on various syntactic structures, including center-embedded and right-branching sentences. A s the results show, the behavior of the network closely resembles the pattern of experimental data, both in yielding superior performance in right-branching structures (compared with center-embeddings), and in processing center-embeddings better when they involve semantically constrained lexical items. This suggests that the recurrent network may provide insight into the locus of similar effects in humans.

One example of linguistic productivity that has been much discussed in the developmental literature is the verb inflection system of English. Opinion is divided on the issue of whether regular^rregular distinctions in surface behavior must be attributed to an underlying distinction in the mechanisms of production. Looking at the course of historical development in English, the current paper evaluates potential shortcomings of two competing approaches. T w o sets of simulations are presented. The first argues that a single-mechanism model offers a natural account of historical facts that would be problematic for a dual mechanism approach. The second addresses a potential problem for a single-mechanism account, the question of default behavior, and demonstrates that even in the absence of superior type frequency a network is capable of developing a "default" category. W e conclude that the single network account offers a more promising mechanism for explaining English verb inflection.'

This paper describes a technique developed for modeling historical change in connectionist networks, and Wefly reviews previous work applying that technique to the problem of the historical development of the regular verb system from early to late Old English. W e then broaden the scope of the simulations and ask what the effect would be of having such changes occur in a single mechanism which is processing both the regular and inegular verbs, and extended over a longer time period. A s w e shall see, the results are highly consistent with the major historical developments. Furthermore, the results are readily under- stood in terms of network dynamics, and provide a ratio- nale for the shape of the attested historical changes.

We present a connectionist model of event knowledge that istrained on examples of sequences of activities that are notexplicitly labeled as events. The model learns co-occurrencepatterns among the components of activities as they occur inthe moment (entities, actions, and contexts), and also learns topredict sequential patterns of activities. In so doing, the modeldisplays behaviors that in humans have been characterized asexemplifying inferencing of unmentioned event components,the prediction of upcoming components (which may or maynot ever happen or be mentioned), reconstructive memory,and the ability to flexibly accommodate novel variations frompreviously encountered experiences. All of these behaviorsemerge from what the model learns.