Trends in the Study of Pavlovian Conditioning

These introductory comments for the special issue on Pavlovian conditioning begin with a brief review of Rescorla’s (1988) influential paper outlining how trends in the study of Pavlovian conditioning changed the dominant view from an analysis of simple associations to an information seeking approach. I argue that current research trends in Pavlovian conditioning suggest possible ways of integrating the information-seeking approach with a more embodied, evolutionary approach. I briefly consider four trends reflected in the following papers that potentially add to our appreciation of how ecology and evolution are involved in conditioning: (1) Exploring an adaptive basis for Pavlovian conditioning; (2) Clarifying the relation of Pavlovian and operant conditioning; (3) Relating preexposure and sensory preconditioning to the organization of exposure learning; and, (4) Integrating models of Pavlovian conditioning. Copyright Information: All rights reserved unless otherwise indicated. Contact the author or original publisher for any necessary permissions. eScholarship is not the copyright owner for deposited works. Learn more at http://www.escholarship.org/help_copyright.html#reuse International Journal of Comparative Psychology, 2004, 17, 119-130. Copyright 2004 by the International Society for Comparative Psychology Trends in the Study of Pavlovian Conditioning William Timberlake Indiana University, U.S.A. These introductory comments for the special issue on Pavlovian conditioning begin with a brief review of Rescorla’s (1988) influential paper outlining how trends in the study of Pavlovian conditioning changed the dominant view from an analysis of simple associations to an information seeking approach. I argue that current research trends in Pavlovian conditioning suggest possible ways of integrating the information-seeking approach with a more embodied, evolutionary approach. I briefly consider four trends reflected in the following papers that potentially add to our appreciation of how ecology and evolution are involved in conditioning: (1) Exploring an adaptive basis for Pavlovian conditioning; (2) Clarifying the relation of Pavlovian and operant conditioning; (3) Relating preexposure and sensory preconditioning to the organization of exposure learning; and, (4) Integrating models of Pavlovian conditioning. An influential paper by Rescorla (1988), published sixteen years ago in the American Psychologist, reviewed a variety of evidence that Pavlovian conditioning is considerably more complex than had been commonly assumed. In the traditional view, repeated contiguous pairings of a CS (conditioned stimulus, a neutral stimulus) with the US (unconditioned stimulus, a reward) produced a simple association between the two events. This association was typically expressed as a conditioned response to the CS that strongly resembled the unconditioned response to the US. Rescorla argued that Pavlovian conditioning was better viewed as reflecting the efforts of an information-seeking organism that produced sophisticated representations of the world based on logic, perceptual relations among events, and preconceptions. Rescorla cited three classes of evidence supporting this expanded view of conditioning. Two classes dealt primarily with informational and representational characteristics, while the third concerned the form of the conditioned response. (1) The first class of evidence concerned the circumstances producing conditioning, including: the critical importance of a contingency relation between the CS and US (rather than simple contiguity) and the unequal associability of particular CS-US combinations; (2) The second class of evidence concerned the complexity of what is learned, including: relations to the context, within event relations, and hierarchical and “competitive” relations among predictive stimulus elements. (3) The third class of evidence pointed out the failure of traditional Pavlovian theory to account for the varied nature of the conditioned response, including: the frequent lack of resemblance between the unconditioned response to the US and the conditioned response to the CS, the emergence of different CRs to different CSs predicting the same US, and the ability of a CS presentation to influence multiple forms of operant behavior. Preparation of this manuscript was supported in part by NSF Grant IBN99-17175. Thanks to Todd Schachtman for asking me to write this introduction, then commenting and proofreading beyond the call of duty, and to Gary Lucas for conversations. Comments concerning this article should be addressed to William Timberlake, Department of Psychology, Indiana University, 1101 E. 10 St., Bloomington, IN 47405, U.S.A. (timberla@indiana.edu).

Rescorla's conceptual work, models, and experimental data have strongly influenced the way Pavlovian conditioning is viewed and, thus, the kinds of questions asked and the interpretations offered for results. Researchers have developed more specific characterizations of the mechanisms involved in encoding, storing, and retrieving by using increasingly complex training and post-training manipulations and tests (e.g., Rescorla, 2001). The papers in this issue (Schachtman, 2004) can be seen as reflecting several current trends in the study of Pavlovian conditioning leading toward more extensive grounding of information-seeking mechanisms in specific evolved functional mechanisms selected in particular ecological contexts (see Rozin & Schull, 1988;Shettleworth, 1998).
In a general version of such an ecological-evolutionary approach, Pavlovian conditioning is presumed to occur through interaction of its procedures and environments with repertoires of perceptual-motor mechanisms and their related motivational states distributed according to the spatial-temporal patterns of USs and CSs (Domjan, 1994;Fanselow, 1994;Timberlake, 1994Timberlake, , 2001; see also Konorski, 1967;Suboski, 1990). Basically, cues proximate to a US (on the basis of learning and/or evolutionary selection) have privileged access to and differentially refine a US-related focal search repertoire, while less proximate but still predictive cues have access to and refine the expression of a more general search repertoire.
Given this view, the sophisticated representations and logic attributed to organisms exposed to Pavlovian conditioning paradigms should be at least partly the product of relatively specific mechanisms with encapsulated characteristics narrowly tuned to particular environmental information interacting with the experimental apparatus and procedures. For example, such specific mechanisms appear to be used by humans in reasoning about causation (e.g., Todd & Gigerenzer, 2003). It follows that close examination of particular examples of Pavlovian conditioning should reveal limits and specificity in processing that can be traced to phylogenetic history and selection for particular circumstances and functions.
It is worth noting that the iterative process of designing and redesigning experimental procedures and apparatus to provide appropriate environmental support for vigorous and reliable learned behavior can easily obscure the specificity of mechanisms involved in Pavlovian conditioning. Well-tuned combinations of mechanisms, procedures, and environments help us to conceptually analyze and experiment with learning at an abstract, general level (see Timberlake, 2001Timberlake, , 2002b, but tend to isolate experimenters from specific characteristics that bear on ecology and evolution. The papers in this special issue can be seen as reflecting four potentially related trends in Pavlovian conditioning: (1) Exploring an adaptive basis for Pavlovian conditioning; (2) Clarifying the relation of Pavlovian and operant conditioning; (3) Relating preexposure and sensory preconditioning to the organization of exposure learning; and, (4) Integrating models of Pavlovian conditioning.

Exploring an Adaptive Basis for Pavlovian Conditioning
The idea that Pavlovian conditioning produces adaptive "preparatory" responses dates back to at least the experiments of Zener (1937) on the functional contribution of preparatory responses in facilitating attention to and readiness for the US. The work of Hollis (see 1997Hollis (see , 1999 for summaries) has tested the adaptive advantage of Pavlovian conditioning in ecological problems simulated in the laboratory, such as territorial defense and mating behavior in blue gouramis. For example, she has shown that fish receiving pairings of a predictive cue with the US of access to a territorial rival are better at defeating that rival than fish receiving unpaired presentations of a CS and access to the rival. Also, the sexual encounters of blue gouramis produced more offspring if they were signaled rather than unsignaled.
In a paper in this issue (Hollis et al. 2004), Hollis and her coworkers extend their work to study the function of Pavlovian conditioning in two social species that form dominance orders. Working with green anole lizards and fire mouth cichlids, they found that subdominant anoles, able to predict the appearance of food, stole food more successfully from their rivals, while the Pavlovian prediction of food for the dominant cichlids allowed them to be more effective in defending food from those less dominant. Akins (2004), in her paper on the role of Pavlovian conditioning in sexual behavior summarized considerable evidence for the adaptiveness of Pavlovian conditioning in encounters between male and females. For example, Domjan, Blesbois, and Williams (1998) showed that male quail for which access to a female was signaled produced more semen and spermatozoa than an unpaired control group. Gutierrez and Domjan (1997) showed that females for whom a signal predicted the appearance of a male partner showed increased receptive squatting when the signal subsequently was presented in the presence of the male partner.
A frequently mentioned difficulty with focusing on the adaptive nature of Pavlovian conditioning is that Pavlovian procedures can easily result in no immediate advantage in the survival or reproduction of an organism, or even produce clearly maladaptive behavior (see Holt, Green, & Muenks, 2004). However, Hollis et al. (2004), point out that all adaptive mechanisms can produce maladaptive results in some circumstances (e.g., consider that the territorial behavior of male lions selected to promote access to reproductive females also frequently leads to the male's death, as can histamine reactions selected to protect the body from intruding substances).
The key to an adaptive approach to Pavlovian conditioning is to establish the nature of the mechanisms of perceptual-motor learning and motivational organization that have been selected in particular sets of circumstances based on their "proper function" in enhancing survival through learning with respect to goals like feeding, avoiding danger, and reproducing (Millikan, 1984;Timberlake, 1994). That the function that facilitates selection of particular learning mechanisms can be altered and subverted by changes in laboratory circumstances is not surprising. The point is that these functional mechanisms and structures exist and underlie the processes of learning.
Establishing the characteristics of underlying perceptual-motor mechanisms and motivational states facilitates understanding and predicting specific characteristics of Pavlovian conditioning, including optimal CS-US intervals for a CR and different CRs to different CS-US intervals (see the heart rate work of Schneiderman, 1972, and the work on predatory behavior of rats to a moving CS by Timberlake, Wahl, & King, 1982). It is worth noting that although Pavlovian conditioning may sometimes approximate optimal outcomes, there is typically greater selection pressure on eliminating mechanisms that lead to death and/or failure to reproduce, than on mechanisms that produce optimal outcomes in multiple circumstances.
The focus of an adaptive approach on the specific perceptual motor organization and motivational states characteristic of a particular system of behavior can produce a defensible framework for the comparative analysis of forms of learned behavior. Akins (2004) points out problems confronting the integration of research on humans and nonhuman animals in the area of sexual behavior. The human studies on the relation between Pavlovian conditioning and maladaptive sexual behavior are necessarily limited and often not well done. The studies of nonhuman animals provide considerable information about mechanism, but these studies are not well coordinated with clinical circumstances.
A strategy in connecting these two literatures would be to relate each study to the specifics of its species' sexual behavior system, including underlying motivational organization and perceptual-motor mechanisms. It may be especially relevant to consider that different CSs are likely to engage different perceptualmotor structures, and that short versus long CS-US intervals are likely to produce different results, with short CSs drawing from a focal search repertoire and long CS-US intervals drawing from the general search repertoire.

Clarifying the Relation of Pavlovian and Operant Conditioning
Pavlovian and operant conditioning have traditionally been considered two fundamental and separate learning processes that (along with habituation) are responsible for the effects of experience on behavior. Ambiguities about their distinctiveness in terms of procedures and underlying mechanisms were largely resolved by the time Hilgard and Marquis (1940) wrote their influential summary of the field of learning. In Pavlovian procedures the experimenter controlled the relation between the CS and the US (and most often measured autonomic responses). In operant conditioning the experimenter controlled what the organism had to do to get the reward (and always measured voluntary responding).
An important example of problems with this relatively neat packaging of tasks, measures, and results was the Brown and Jenkins (1968) discovery of autoshaping-the emergence of apparently voluntary keypecking in pigeons exposed to Pavlovian procedures pairing an illuminated keylight (a CS) with the delivery of grain (the US). Once established, it became obvious that similar effects occurred in other circumstances and for other organisms (Hearst & Jenkins, 1974;Timberlake & Grant, 1975). Several solutions were offered to maintain the traditional separation of operant and Pavlovian learning. An important possibility was that emergent keypecking under Pavlovian contingencies was actually reinforced as an operant by the delivery of food. This was a familiar and appealing explanation that, unfortunately, was made untenable by Williams and Williams's (1969) demonstration that an omission schedule turning off the keylight and preventing food delivery failed to eliminate pecking. Schwarz and Gamzu (1977) offered another possibility for rescuing the distinction between operant and Pavlovian conditioning by distinguishing between the topography of operant and Pavlovian key pecks. Holt, Green, and Muenks (2004) reviewed this work and concluded that the data supporting this view were inconclusive. Holt et al. (2004) offered another potential way to distinguish oper-ant and Pavlovian processes by relating them to different patterns of responding under different component lengths in a two component multiple VI (variable interval) feeding schedules in which a relatively dense VT (variable time) schedule was added to one of the components. With very short 10-s components, adding the VT schedule to one component increased pecking at the beginning of that component. The authors called this a biological effect, apparently produced by Pavlovian stimulus-reward contingencies. With very long schedule components (20 min), adding the short VT to one component decreased overall peck rate in that component. The authors labeled this an economic (regulatory) effect, presumably produced by operant response-reward contingencies.
The use of different response patterns under complex schedules to designate Pavlovian and operant conditioning is interesting and clever, but it may not be as generally helpful as the authors may have hoped. For example, I suspect that by experimenting with different VR (variable ratio) sizes and component lengths, one could produce results similar to the VI-VI with added VT schedules above. The short component condition should show anticipatory responding for the component with the shorter VR, and the long component condition would show regulatory (lower rate) responding in the component with the shorter VR. Such malleability of patterns of response rates (see also Peden & Timberlake, 1987) raises questions about the possibility of uniquely relating increases and decreases in response rate as due to different kinds of conditioning.
In short, there are unquestionably differences in operant and Pavlovian procedures, but because we are dealing with a behaving organism the results cannot be as cleanly separated as our procedural definitions. An operant procedure focuses on response-reward dependencies, but it cannot avoid stimulus-reward dependencies introduced by the behavior of the organism. A Pavlovian procedure focuses on stimulus-reward dependencies, but it cannot avoid response dependencies resulting from the organism's behavior. One could incapacitate the organism, but the underlying neural circuitry would still provide a background for response dependences. Further, it is worth considering that these complex interdependencies in learning were probably selected for in natural settings because they promoted adaptive behavior.
Given that an organism brings a common substrate of perceptual-motor and motivational mechanisms to all learning circumstances, an ecologicalevolutionary view suggests we should focus our attention on how the characteristics of these organisms interact with the characteristics of experimental environments and procedures. Whether the procedure is Pavlovian or operant, the result will depend on which perceptual-motor mechanisms are engaged, how relatively open they are to engagement and modification in specific environments, and what motivational systems and states are involved (Timberlake, 2001). The expression of the underlying organization should reflect specific support from the context, the programmed relations among specific environmental cues and specific response contingencies and reward, and regulatory "hill-climbing" tendencies for the motivational systems.
In this view, the results of Pavlovian and operant procedures differ because they typically interact differently with this substrate, not because there are completely separate Pavlovian and operant learning systems. Consider that Pavlovian schedules (the contingent relations between the CS and US) have different effects as a function of different CSs and/or different CS-US intervals. Different operant schedules produce markedly different response patterns as a function of requirements and payoffs. On the whole, it seems arguable that Pavlovian and operant conditioning are not independent building blocks of learning, but represent experimenter measurement of different expression of learning resulting from the way our procedures ultimately contact intercommunicating substrates of the brain.
Applying such an approach to the results of Holt et al. (2004), it might be argued that the key to the different effects was likely to be a predictive focal search state tied to the short component schedules, while a general search state was dominant in the long components schedules. Focal search behaviors tend to be strongly related to specific target cues proximate to food, whereas general search behaviors tend to be more related to distributing time in preferable food locations.
Finally, Zentall's (2004) work nicely follows up this discussion by focusing on how to clarify the intersection of Pavlovian and operant procedures. Zentall notes that ordinary operant discrimination training can produce notably different effects on the assessed value of the S+ and the S-. Initially, the S-against the background of the S+ and the context appears more valued than if it had been presented alone. Later the S-appears less valued relative to the S+ and the context, than if no S+ had been present. Zentall argues that simultaneous discrimination may provide a useful framework for evaluating several Pavlovian conditioning phenomena, including higher order conditioning, within-event conditioning, postconditioning devaluation effects, inhibitory conditioning, potentiation, and perhaps overshadowing.

Preexposure, Sensory Preconditioning, and the Organization of Exposure Learning
The indirect effects of preexposure to stimuli and environments on Pavlovian conditioning (i.e., CS preexposure, US preexposure and preconditioning of two CSs) have historically received less attention than directly modifying behavior through CS-US pairings. However, Lubow (1989) and others documented the decremental effects of CS-preexposure on subsequent conditioning, ruled out simple habituation to the CS as a cause, and gave the decremental result a name (latent inhibition) and an explanation compatible with an information-seeking model. As could be expected from an ecological-evolutionary view, though, further work has begun to reveal multiple effects, and further questions. One such question was raised by Boughner, Thomas, and Papini (2004). They present data showing that the typical control group for latent inhibition (simple exposure to the experimental environment) can have excitatory effects on subsequent autoshaped conditioning in rats when the size of the US is large, or the trials are massed with short intertrial intervals. Further, they show that the effects of preexposure are related to the response measured.
In a seeming parallel to the decremental effects of CS preexposure on subsequent conditioning, US preexposure before training also seems to inhibit subsequent CS-US training (e.g., Tomie, 1976); however, the effect seems differently mediated. For example, preserving the timing of the US presentation from preexposure to conditioning appears important (Goddard & Jenkins, 1988), but preserving the context does not, at least in the case of a shock US (Williams & Lolordo, 1995). Jennings and Kirkpatrick (2004) further explored the conditions for US preexposure decrement by manipulating the CS duration and a 31.5 min presession waiting period in the context. They interpret their results as showing that US preexposure effects occur when relatively weak conditioning would be expected, e.g., a 31.5 min presession extinction, and a CS length nearly the size of the intertrial interval. Goddard (2004) showed that the presentation of a single US at the 10-min mark of a 14-min session for 20 days interfered with the subsequent conditioning effect of five days on which the single pellet was followed by 3 pellets presented 20 s later. Based on these data (and a significant amount of other research he reports using a similar procedure), Goddard argued that the reason that the 20 daily trials of preexposure interfered with learning about the subsequent US presentations was because in pretraining the US delivery signaled a US free period, thereby producing inhibition of US approach. However, based on the classic surprise argument, if the rats had learned to expect no pellets following a single pellet at 10 min, the actual delivery of 3 more pellets should have been a surprise and therefore should have facilitated (or at least not reduced) conditioning to the new pellets. Alternatively, it seems possible that the single pellet entrains wider foraging because the single food pellets is never repeated at that location during a session.
In contrast to the generally inhibitory effects of preexposure, the preconditioning procedure of presenting two CSs, sequentially or simultaneously prior to training, puzzlingly appears to produce a form of excitation. Following multiple pairings of the two preconditioned CSs, one of them is paired with a US for multiple trials. Then the second preconditioned CS is simply tested for responding in comparison to a neutral CS. Depending on the temporal position of the second preconditioned CS relative to the first in pretraining and the temporal relation of the first preconditioned CS relative to the US in training, excitatory conditioning clearly can be shown to the second preconditioned CS. For example if the second CS precedes the first in pretraining, and the first CS is backward paired with a US, the second CS shows excitatory conditioning in test as though it had been presented before the US (Barnet & Miller, 1996).
Miller and his colleagues have explained such phenomena by arguing that the organism forms a "temporal map" of the two CSs in preconditioning, and during conditioning the US is placed "on" that temporal map based on its relation to the first CS. Thus, in the previous example the US is placed before the first CS and, therefore, after the second CS. It is not clear whether such a map simply records sequences of events, or also records their absolute and/or relative duration.
In addition to providing evidence for a temporal/sequential map, the preconditioning procedure raises the question of why repeatedly presenting two stimuli (as opposed to one) in a context without a US does not appear to produce subsequent latent inhibition. The issue can not be resolved definitively for the preconditioned stimulus that is paired with the US because most researchers have not used a control to test for latent inhibition. What seems clear, though, is that there is no inhibition of responding to the second preconditioned stimulus when compared to responding to a neutral stimulus. Instead it appears that the US-based conditioning to the first preconditioned stimulus is passed on to the second preconditioned stimulus (in a temporally appropriate manner) in the absence of pairings with the US.
Whether the apparent absence of latent inhibition is created by repeated exposure to two cues rather than one, or to some other issue, remains to be worked out. Whatever the answer, the current work on exposure learning raises several additional issues, including that surprise may play a different role in patterns of US presentations alone, and the apparent role of exposure in developing temporal, sequential, and spatial maps.
The complex combination of outcomes in different forms of exposure learning suggests the potential importance an ecological-evolutionary attempt to understand the basic stimulus encoding organization animals have available for learning about temporal and spatial relations among stimuli through simple exposure. The evidence of such preorganization also indicates the importance of relating learning tendencies of a species to the apparatus procedures we use to test them. For example, my colleagues and I (Timberlake, 2002b) provided a good deal of evidence that the maze performance of rats reflects the interaction of a number of exposure-activated mechanisms related to trail following and learning about the spatial environment.

Integrating Models of Pavlovian Conditioning
After years in which the Rescorla-Wagner model dominated Pavlovian conditioning (Rescorla & Wagner, 1972), Pearce's (1994) configural theory and Kruschke's (1992) ALCOVE model helped focus attention on within-cue associations. SET (Gibbon & Balsam, 1981) and then RET (Gallistel & Gibbon, 2000) focused attention on timing and relative spacing of CS and US events. The comparator hypothesis (Miller & Matzel, 1988) and its multiple offspring (e.g., Denniston, Savastano, & Miller, 2001) compelled attention to the experimental context. The behavior systems approach took on predictions of CR form as a function of CS characteristics and CS-US intervals (Domjan, 1994;Fanselow, 1994;Timberlake, 1994;Timberlake & Lucas, 1989). In comparing these models (with a few bridging exceptions, such as Wagner's SOP and AESOP models- Wagner & Brandon, 1989), there appear to be large gaps between models that focus on associative elements, models that deal with timing elements, and models that focus on behavioral topography and function. This lack of integration seems an appropriate area for further work, perhaps in terms of testing real-time embodied simulations (see Timberlake, 2002a).
A less radical approach, often missing in the modeling of Pavlovian conditioning, is the application of multiple models to the same data. Investigators often tend either to ask a particular model to make a prediction and then test its accuracy, or contrast two models in a "shoot out" to see which is "correct." The paper by Mehta et al. (2004) specifically tested the ability of four models, Rescorla-Wagner (1972); the computational comparator hypothesis (Murphy, Baker, & Fouquet, 2001ab); Pearce's (1994) configural theory, and the extended comparator hypothesis (Denniston, Savastano, & Miller, 2001), to account for differences between separate and simultaneous training of two predictive cues.
A second frequent neglect in modeling is a thorough exploration of the determinants of a result combined with an analysis of alternative models that goes beyond picking the best fitting one to an understanding of what parts of each model do the major work in fitting results and which parts make fitting difficult.
-127-The paper by Jennings and Kirkpatrick (2004) begins an empirical exploration of the determinants of the US preexposure effect, while considering multiple models, each of which fits only aspects of the data. Filling out a "matrix" of circumstances that produce US preexposure appears likely to clarify what specific aspects of the different models does the fitting work and what parts do not help.
Also on the point of empirical approaches, Miller and Escobar (2004) summarize extensive empirical regularities across the whole of Pavlovian conditioning procedures. Their effort seems worthwhile for two reasons: they provide a background of empirical learning regularities that can serve as check list for any presumably general theory; and, they invite some novel classifications of phenomena based on low level procedural criteria that often relate well to systematic differences in effects. Such regularities can lead to separating learning types that have historically been treated together and incorporating otherwise outlying examples into a common category. This can lead both to important insights and to occasional "fool's gold" as categories are eliminated or reworked. Nonetheless, that Miller and Escobar have accounted for and "tagged" a great amount of data should be an advantage for subsequent modeling.
Finally, Miller and Escobar (2004) cite Thorndike's work as an example of using theory to define learning and Skinner's work as an example of using empirical procedures. I do not perceive this separation as quite so clean. Skinner's empirical analysis of an operant contingency bears points of resemblance to Thorndike's empirical definition of reinforcers as something the organism will approach. The major concepts that make up Skinner's contingency (reinforcers, operants, and discriminative stimuli) are actually defined only after the fact of reinforcement (Timberlake, 2003). Further, there are a variety of low-level theories present in operant conditioning, such as the notion of contiguous strengthening illustrated in the concept of superstitious behavior. Because similar arguments pointing to a conceptual basis can be brought with respect to almost any welldeveloped empirical approach, the healthiest stance appears to me the continued interaction of conceptual-theoretical and empirical analysis

Conclusions
Some current trends in Pavlovian conditioning research appear to be moving toward a more specific embodiment of conditioning in the evolutionary and ecological biology of the subject. These trends show promise in figuring out how to understand adaptiveness, predict the form of the contingent response, analyze the organization of exposure learning, clarify the relation of Pavlovian and operant conditioning, and integrate different models of Pavlovian conditioning. It should also ultimately provide a more explicit interface with neurophysiological analyses (e.g., Steinmetz, Kim, & Thompson, 2003).