Do honey bees (Apis mellifera) form cognitive representations of unconditioned stimuli?

Previous research studies looking at expectancy in animals have used various experimental designs that focused on appetitive and avoidance behaviors. In this study, honey bees ( Apis mellifera ) were tested in a series of 3 proboscis extension response (PER) experiments to determine to what degree honey bees form a cognitive-representation of an unconditioned stimulus (US). In the first experiment, bees were presented with either a 2-s sucrose or honey US appetitive reward, and the proboscis-extension duration was measured for each scenario. The PER duration was longer for the honey US even though each US was presented for just 2 secondss. Honey bees in the second experiment were tested during extinction trials on a conditioned stimulus (CS) of cinnamon or lavender that was paired with the same US (either sucrose or honey) used in the acquisition trials. The proportion of bees showing the PER response to the CS along with the duration of the proboscis extension was recorded for each of the extinction trials and US scenarios. Neither measure differed between the honey and sucrose US scenarios. In Experiment 3, bees were presented with a cinnamon or lavender CS paired with either the honey or sucrose US in a set of acquisition trials, but, here, the US was not given until after the proboscis was retracted. The PER duration after the CS and again subsequent to the US was recorded. While the PER duration after the US was longer for honey, the PER duration after the CS did not differ between the honey and sucrose US. The results from the third experiment indicated that the bees in this experiment did not use the learned CS and US pairings later on to anticipate the US based on the paired CS. Overall, bees in this study demonstrated an inability to solve the problem presented to them, suggesting possible limitations in honey bee’s cognitive abilities.

The level of detail in a memory defines the information content, or cognitive representation (Glass & Holyoak, 1986), held by an animal. The memory representation subsequently defines the manner in which a subject responds to a stimulus, sets limits on the ability to solve problems, and even leads to differences in anticipatory behavior. Problem solving studies have been effective in advancing our understanding of cognitive representations in humans (Seegelke & Schack, 2016), but little has been done to examine the differences that exist across animal taxa, particularly in invertebrates. The purpose of this study was to determine if bees form more than a simple cognitive representation of an unconditioned stimulus by testing for reward using specific aspects of anticipatory behavior.
The rationale behind the work presented here are the experiments performed by Konorski (1967), which suggested that an unconditioned response can be partitioned into consummatory and preparatory responses. This partition suggests that one method of studying whether invertebrates form representations of cognitive events is to create a situation where the unconditioned response differs between two unconditioned stimuli. If the response differs between the two unconditioned stimuli, a representation of the stimuli is suggested. One of the best examples of this was provided by Moore (1973) who demonstrated that the topography of a pigeon's key peck differed based on whether the target signaled food or water.
The expectation of upcoming events is clearly shown in anticipatory behaviors, which are found in a wide variety of vertebrate and invertebrate taxa, as seen in the results of classic, discriminant, and operant conditioning studies (e.g., Robbins et al., 2001). Anticipatory behavior has been linked to physiological changes in cichlid fish (Galhardo et al., 2011), and, in some vertebrate species, anticipation can be based simply on elapsed time rather than a traditional conditioned stimulus (e.g., scent, color, or sound). When anticipation is based on elapsed time, researchers have found that anticipation is closely linked to genes involved in time perception (Feillet et al., 2006). However, an anticipatory response by itself tells researchers little about the cognitive representation of a stimulus. Under these circumstances, differences in anticipatory behavior are illuminating. For example, domestic fowl discriminated among sound cues that were paired with positive, neutral, and negative events and responded with different anticipatory behaviors for each of the different cues (Zimmerman et al., 2011).
The question naturally arises whether invertebrates, and in particular insects, can also form more than a very basic cognitive representation of a stimulus. In terms of insects, pollinators are a likely organism to form representations of rewards based on the tasks they routinely perform (Leadbeater & Chittka, 2007). An ideal pollinator to explore insect cognitive representation is the honey bee, and the literature is rich with examples of both associative and nonassociative learning scenarios for this organism (Abramson, 1990;Black et al., 2018;Gerber et al., 1996). This research stems back to the work of von Frisch on the appetitive conditioning of free-flying bees to targets (Abramson, 1990(Abramson, , 1994 and, more recently, on the use of free-flying foragers visiting artificial flower patches (e.g., Giray et al., 2015;Sanderson et al., 2006). When bees forage, they store the locations and best visiting times associated with the most profitable flower patches (Chittka, 2017;Giurfa, 2007). They are also able to use the fastest route to get to the location and remember previous experiences when foraging, such as threats that they encounter (e.g., various predators; Chittka, 2017;Leadbeater & Chittka, 2007). Further, bees store both short-and long-term memories (Giurfa, 2007). Recent literature provides evidence that bees possess several sophisticated cognitive abilities in at least three conceptualization/categorization areas. This would be in line with bees having nontrivial cognitive representation abilities, discussed below.
Researchers have tested bees' cognitive abilities using several different types of methods Avarguès-Weber, Portelli, et al., 2010;Giurfa, 2001). The results from these studies provide evidence of the honey bee's sophisticated cognitive abilities. Bees have the ability to understand the relationships of similarity and difference (Giurfa et al., 2001), to follow rational rules (Avarguès-Weber, Portelli, et al., 2010), and to manipulate relational concepts (Avarguès-Weber, Dyer, & Giurfa, 2010). Giurfa and colleagues (2001) trained and tested bees using a delayed matching to samples task (bees respond to matching stimuli) and delayed nonmatching to sample task (bees respond to different stimuli) and found that the bees would continue to choose the appropriate behavior (i.e., matching or nonmatching) for novel situations. These findings indicate bees can use their abilities to identify similar stimuli and discriminate between different stimuli to problem solve. Future studies by researchers were conducted to determine whether bees could solve complex problems.
Avarguès-Weber, Dyer, and  and Avarguès-Weber, Portelli, et al. (2010) conducted a series of experiments to test bees' cognitive problem-solving abilities using rational rules ("above" and "beyond") and configural designs. In Avarguès-Weber, Dyer, and  first experiment, bees were trained to choose a target (e.g., lines, circle, triangles) above or below a referent (i.e., a black bar). Bees in the second experiment were presented with novel situations where they had to use the previously trained spatial concept. Bees in the second experiment used the previously learned "above" and "below" rules in scenarios where variables such as the distance between the target and the referent bar differed. The ability to follow rational rules such as "above" and "below" was previously thought to be exclusive to humans and nonhuman primates (Avarguès-Weber, Dyer, . The results from this study indicated that bees have the ability to manipulate relational concepts and can apply them to novel scenarios as illustrated by the bees in this study . In the second study, Avarguès-Weber, Portelli, et al. (2010) found that bees were able to successfully use configural visual-processing of different facial feature, dot, and line compositions to categorize face-like and non-face-like images. The bees categorized face-like and non-face-like stimuli made out of various stimuli (e.g., dots, lines, nose, mouth, and eyes) using basic relationships (such as how the features conformed to the standard facial arrangement) between face-like and non-face-like features (Avarguès-Weber, Portelli, et al., 2010). Bees completed a series of complex recognition tasks in which they had to determine whether or not the stimuli conformed to the typical arrangement and location of a face (e.g., two eyes, a nose, and mouth; Avarguès-Weber, Portelli, et al., 2010).
Combined, these studies suggest that bees have complex cognitive abilities and that they should be able to anticipate and form a cognitive representation of a paired reward based on the presentation of a conditioned stimulus (CS). The studies that were mentioned above illustrate the different cognitive abilities of the bee, which indicates that bees are able to apply learned information to new situations. In our study, we tested the bee's ability to transfer previously learned information (CS-unconditioned stimulus (US) pair) to predict a US reward.
Work suggesting bees may have cognitive representations beyond acceptable appetitive rewards stems from two types of experiments. In the first type of experiment, responses to two-choice reward-quality difference problems were used as a derivative function of perceived reward, and the integral of that curve was used to formulate a functional response graph relating perceived reward to actual reward. Results suggested a linear functional response with honey bees overestimating higher quality rewards (Sanderson et al., 2013). Direct comparison of appetitive choices is an important step in the integration of information from a range of rewards by individuals because the cognitive representation of reward differences can be on a quantitative scale rather than nominal, acceptable or not (Sanderson et al., 2013).
Second, Gerber and colleagues (1996) found evidence showing that what honey bees learn under field conditions can be transferred to a laboratory situation in which bees are harnessed while undergoing classical conditioning of the proboscis conditioning response. This experiment was replicated using a shuttle box situation in which the bee's choice of spending time on one or two simultaneously presented colors was influenced by the bee's previous experiences with visual cues (Black et al., 2018). The results from this experiment indicate that the cognitive representation of the problem was complex enough that it could be used in a very different scenario.
Nevertheless, there are major areas of cognitive representation that appear to be minimal or nonexistent in honey bees. Cognitive representation of time by honey bees seem to be much poorer than initially suspected (e.g., Butler, 1945). When honey bees are presented with a fixed-interval schedule of reward (10 or 15 s), they do not come under temporal control (Craig & Abramson, 2015). Bees do not appear to anticipate the reward presentation like some vertebrate species do (e.g., Ferster & Skinner, 1957;Skinner, 1938). Rather than returning to the field-feeding stations in mass at the time of reward presentation, foragers respond in mass to the CS odor brought back to the hive by a single bee due to their cognitive representation of time (Farina et al., 2005;Reinhard et al., 2004;Wells & Rathore, 1995;Wenner et al., 1969). Poor (or lack of) time perception limits anticipatory abilities and may be especially problematic for the development of rudimentary cognitive representation of anything involving time as a component of behavior.
Another area where honey bees seem to lack cognitive representation involves Pavlovian conditioning using the removal of an odor from the environment (i.e., the removal of a continuous odor, CS, signaling the reward coming). Bees are able to associate the presentation but not the removal of the odor with a reward . Similar results using a signal avoidance paradigm also provide evidence that honey bees do not have the ability to use the removal of a cue in learning to expect a punishment (Abramson, 1986).
Finally, performance in a feature-positive/feature-negative study questions the expectancy abilities of honey bees. A feature-positive condition is when a stimulus is added to signal the reward, and a feature-negative condition is when a feature is removed to signal the reward. Bee success rates for the feature-negative trials were approximately 64% and approximately 60% for the feature-positive trials, which is a little better than chance (i.e., 50%). These findings suggest that the foragers did not retain information in detail from previous trials and that the bees had to reassess the problem during each trial .
In a series of experiments, we tested honey bees to determine whether they could form a cognitive representation of different rewards through their anticipatory behavior. More specifically, we used rewards that took different lengths of time to consume based on both viscosity and presentation. The duration of the proboscis extension reflex (PER) in harnessed bees was used as a measure of representation. The use of the PER with harnessed bees has extended our knowledge of bee learning through classical conditioning scenarios, including conditioning cue discrimination tasks (Abramson, 1990;Van Nest, 2018), and the anticipation of an appetitive reward following a CS, which appears to be ubiquitous across subspecies of honey bees (Abramson et al., 2008). These experiments are an early step in determining the bounds of cognitive representations in insects.

Study Subjects
Each study consisted of 20 honey bees (Apis mellifera ligustica). Bees were captured in a glass vial at an artificial feeder and harnessed in tubes following the PER protocol of Abramson and colleagues (1997). Bees were fed 1.8 M sucrose solution until satiated and then were left unfed for 24 hr. Animals that did not actively extend their proboscis when presented with sucrose stimulation during a pre-test were excluded.

Reward: Unconditioned Stimuli
The experiments were designed to test for cognitive representation of different US rewards through anticipatory behavior. To that end, we envisioned a reward scenario where the US differed in the time it took bees to consume due to viscosity, presentation, and quality. The first US, honey (~3.10 kcal/ml), was presented as a viscous droplet. The second US was a 1.0 M sucrose solution (~0.45 kcal/ml) infused in a piece of filter paper. This reward duet also had the advantage in which the honey US had taste and nutritional components, which might make it easier to form a cognitive recognition of the US differences. Correspondingly, we used the proboscis extension time (PER duration) as a measure of the cognitive representation of anticipatory behavior.

Experiment 1
The first experiment served as a control to show that the PER duration for the honey US was longer than for the sucrose US. Harnessed honey bees were presented for 2 s with either a viscous honey droplet or a nonviscous sucrose solution (filter paper strip containing 1.0 M sucrose). The difference in viscosity and presentation was predicted to make the honey more difficult for bees to consume, which we expected would be reflected in a difference in proboscis extension time. Each honey bee was given 24 trials: 12 honey trials and 12 sucrose trials. The length of time in seconds from the removal of the US to the time the proboscis was retracted was measured for each test trial. This is the unconditioned response proboscis extension reflex duration (UR-PER duration). Half of the bees received the 12 honey US trials first (10 bees), while the other half of the bees received the 12 sucrose US trials first (10 bees). The intertrial interval (ITI) was 5 min in each case.

Experiment 2
The second experiment tested for differences in anticipatory behavior following a CS based on the US (reward) used in the acquisition trials. Honey bees (N = 20) were presented with a CS of either cinnamon or lavender scent for 3 s in all 24 trials. The first 12 trials were acquisition trials in which the CS was followed by a 2 s presentation of the US. For each CS, half of the bees received the honey US, and the other half received the sucrose US (lavender-honey N = 5, lavender-sucrose N = 5, cinnamon-honey N = 5, cinnamon-sucrose N = 5). The second 12 trials were extinction trials in which the CS was not followed by a US. Each bee continued to receive the same CS as it was given in the acquisition trials. For each of the 12 extinction trials, the length of time in seconds from the removal of the US to the time the proboscis was retracted was measured. This is the conditioned-response proboscis-extensionreflex duration (CR-PER duration). CS scents were used to mitigate differences in PER based on a particular odor.

Experiment 3
The results of the second experiment led us to test whether the CR-PER duration would differ from the immediately following UR-PER duration in each acquisition trial. In fact, we made the honey US presentation longer than that of the sucrose US to accentuate the difference between the alternative US. Thus, Experiment 3 was a set of acquisition trials in which we intentionally gave bees receiving the honey US a longer time to consume the reward than the bees receiving the sucrose US. This is different from both Experiments 1 and 2 in which the US, whether honey or sucrose, was presented for only 2 s. We recorded the CR-PER duration and the UR-PER duration for each trial. Here, for the honey US, the UR-PER duration was defined as the time until the proboscis was retracted minus 2 s, in order to match the UR-PER duration for the sucrose US. We expected a correspondence between anticipatory behaviors (CR) and reality (UR).
Honey bees (N = 20) were presented with a CS of either cinnamon or lavender for 3 s in all 12 trials. The 12 trials were acquisition trials in which the CS was followed by a US. For each CS, half the bees received the honey US, and the other half received the sucrose US.
As in the acquisition trials of Experiment 2, bees were not presented with the sucrose or honey US until the PER to the CS was retracted. Once the CS proboscis response was retracted, then the US was presented. The honey US was presented until the drop was consumed, while the sucrose US (filter paper strip containing 1.0 M sucrose) was presented for 2 s. Both the sucrose US and honey US were presented in front of the test subject prior to allowing the bee to feed, which controlled for the scent of the US. We recorded the UR-PER duration to confirm that they differed between the honey US and sucrose US, as well as in the CR-PER duration. The ITI was 5 min in each case.

Data Analysis
For the first experiment, independent-sample t-tests were used to compare the responses of bees in the honey-US group to that of bees in the sucrose-US group. For Experiments 2 and 3, independent-sample t-tests were used to compare the responses of the bees in the group given the cinnamon and lavender CS within each US (honey and sucrose). After determining that there were no significant differences between the CS stimuli within each US group, the CS data within each US group were aggregated and independent-sample t-tests were used to compare the responses of bees in the honey US group to that of bees in the sucrose US group.

Experiment 1
There was a significant difference in average UR-PER duration between the honey US trials and the sucrose US trials, t(22) = 41.15, p < 0.01. The honey-trials mean UR-PER duration was 17.96 s and the sucrosetrials mean UR-PER duration was 1.32 s. Bees in the honey US trials had a significantly longer average UR-PER duration than honey bees in the sucrose US trial conditions (Figure 1).

Mean (M ± SD) UR-PER Duration of Honey Bees in the Sucrose US and Honey US Trials (Significance level p < 0.01)
Having shown in Experiment 1 that UR-PER duration was longer for the honey US than for the sucrose US, the question became whether a CS paired with the honey US versus sucrose US resulted in a conditioned response consistent with the UR-PER duration of Experiment 1. The proportion of bees showing a PER response in the honey US acquisition and extinction trials closely matched that of the sucrose US acquisition and extinction trials (Figures 2A [cinnamon] and 2B [lavender]). Thus, expectation of an appetitive reward following the CS was not different between bees receiving the honey US versus the sucrose US. When we went on to look at the CR-PER duration during the extinction trials, we found that there was not a significant difference (Figure 3) for the honey US and sucrose US, t(6) = 1.18, p = 0.27. The honey-trials mean CR-PER duration was 1.13 s (SD = 0.17 s), and the sucrose-trials mean CR-PER duration was 1.03 s (SD = 0.18 s). Bees anticipated a US reward, but they did not show evidence of anticipating a specific US reward (e.g., sucrose or honey) based on the presentation of the CS.

The Proportion of Bees showing a PER Response to the Cinnamon (Graph A) and Lavender (Graph B) CS in Acquisition and Extinction Trials
Note. Depicted are honey US trials (solid line) and sucrose US trials (dashed line). A: The proportion of bees showing a PER response to the cinnamon CS for each of the 12 CS-US paired acquisition trials (trials 1-12) and the cinnamon extinction trials (trials 13-24) (M ± SD). B: The proportion of bees showing a PER response to the lavender CS for each of the 12 CS-US paired acquisition trials (trials 1-12) and the cinnamon extinction trials (trials 13-24) (M ± SD).

Experiment 3
Having shown in Experiment 2 that bees did not exhibit significantly different CR-PER durations when the CS was paired with the honey US versus the sucrose US during the extinction trials, a third study looked at the relation between the CR-PER duration and UR-PER duration in the acquisition trials. There was a significant difference in UR-PER duration between the honey US trials compared to the sucrose US trials, t(22) =57.41, p < 0.01. The honey US trials was 21.73 s (SD = 1.52 s), and the sucrose US trials average UR-PER duration was 3.61 s (SD = 0.27 s). This difference is similar to what we recorded in Experiment 1: Bees in the honey US trial conditions had a significantly higher UR-PER duration compared to honey bees in the sucrose US trial conditions (Figure 4). Nevertheless, the CR-PER duration did not differ significantly between honey US and sucrose US trials, t(22) = 1.44, p = 0.17; Figure 4.

Discussion
This work examined whether bees form different cognitive representations of rewards that differed in the length of consumption time based on their viscosity and presentation. The results from Experiment 1 provided evidence that there is a difference in the PER duration between the honey and sucrose US paradigms. Honey bees had a significantly longer average UR-PER duration for the honey US. These results also mirror findings in artificial flower patch work in which different nectar rewards were paired with different flower colors (Wells & Rathore, 1995).
The bees in Experiment 2 did not exhibit behaviors that suggested they expected the honey rather than the sucrose reward when tested in the extinction phase of the experiment. When presented with the CS, the CR-PER response indicated that they anticipated a reward would follow but not that they expected the honey instead of the sucrose reward. The PER duration was the same whether the paired CS-US was honey or sucrose. This finding is consistent with the results of an operant conditioning cap-pushing-response experiment (Abramson et al., 2016). Learning by the autoshaped bees in the cap-pushing experiment did not seem to carry over to the covered-well test treatments.
The results of Experiment 3 showed that the bee proboscis extension CR-PER duration did not differ between honey US and sucrose US acquisition trials, even though it did in the immediately following UR-PER response to the reward. Both the UR-PER durations differed between the honey and the sucrose US rewards. However, the CR-PER response did not differ between the honey US and the sucrose US. Thus, there did not appear to be different cognitive representations of the honey US versus the sucrose US.
Previous research indicates that while honey bees are able to learn through discrimination, they have difficulty with other types of problem solving (Abramson et al., 2008;Abramson et al., 2010;Abramson et al., 2013;Craig & Abramson, 2015). Abramson and colleagues (2008) found that bees were able to discriminate between two odors, which suggests that honey bees might be able to discriminate between positive and negative stimuli. However, when honey bees were asked to choose between a feature-positive and a feature-negative disk, the bees performed a little better than chance . These results are consistent with limited cognitive representation. Abramson and colleagues (2010) also found evidence that honey bees have difficulty problem solving when presented with situations in which a stimulus is removed. Bees were unable to use the removal of a cue as the CS to differentiate between expecting a negative or positive US . In fact, Abramson and colleagues (2010) found that honey bees can learn as a result of a positive stimulus but have difficulty learning from a negative stimulus as well as from the removal of a stimulus. There is evidence to support the presence of behavioral biases based on the visual stimulus presented in aversive situations prior to the experimental sessions (Black et al., 2018). Other species have been shown to use similar complex problem-solving skills; however, they have also had some difficulty adapting these abilities (Cheng, 1986;Grant & Blatz, 2004;Matsuno & Tomonaga, 2006). Some examples of these studies are featurepresent/feature-absent matching in pigeons (Grant & Blatz, 2004), visual searching for stationary and moving objects in chimpanzees (Matsuno & Tomonaga, 2006), and spatial representation tasks in rats (Cheng, 1986).
Previous research provides evidence of behavioral differences in subspecies of bees, suggesting that different subspecies (or species) of bees might differ in their behavior when tested on these experiments (Cakmak et al., 2010). For example, four subspecies of European honey bees were tested for differences in foraging behavior. The degree of flower color fidelity differed among subspecies and was dependent upon which color was favored. The intensity of forager preference for yellow or blue and white flowers varied among the subspecies. However, in a previous Pavlovian conditioning proboscis extension study, no subspecies differences were found (Abramson et al., 2008). It is important to consider both subspecies similarities and differences when studying honey bees .
The results of this study correlate well to findings from a study on interval-timing tests using freeflying bees (Craig & Abramson, 2015). Honey bees were subjected to a fixed-interval reward schedule (15, 30, 60, or 120 s), but no traditional evidence of temporal perception was found for any of the fixed intervals.
In fact, the longer fixed-interval schedules caused all subjects to cease foraging at the station.
Combined, the findings from our experiments provide some answers about the cognitive capabilities of the honey bee. The bees in the first experiment were able to differentiate between the two USs. Bees in the second experiment anticipated a US reward, but their responses indicated that they did not differentiate between the two US rewards. Bees in the third experiment did not form a more complex representation of the US based on the presentation of the CS, indicating that they lack this more complex ability found in humans and some nonhuman animals. This is interesting because other researchers have found that bees can perform different complex problem-solving tasks (Avarguès-Weber, Dyer, Giurfa et al., 2001).

Conclusions
The three studies presented here will help us to understand the cognitive limits of honey bees. Similar to the findings of other studies, honey bees had difficulty with problem solving. Clearly, researchers still have much to learn about the extent of cognitive abilities in invertebrates such as the honey bee.
Although we could not show anticipatory CR-PER behavior with respect to the nature of the reward under the harnessed-bee experimental design, free flying bees may still show cognitive representation as tested in these experiments. Harnessed-bee PER-studies have been extremely valuable when using scent conditioning stimuli; nevertheless, this protocol fails when using color conditioning cues (Niggebrügge et al., 2009). This occurs in spite of how good honey bee color vision is and how well bees use this ability in natural foraging (Niggebrügge et al., 2009). In fact, even seemingly minor changes in experimental scenarios can result in surprising differences in behavior. In a classic example using artificial flower patch studies, whether foragers walk rather than fly from flower to flower can produce stunning differences in cognitive response to reward differences (Wells & Wells, 1984). Frost and colleagues (2012) stated that factors such as the season during which the study was conducted can also impact bee learning. Changes in learning due to seasonal temperature variations have been recorded in bees from outdoor hives (Blažytė-Čereškienė & Skirkevičius, 2006;de Jong & Pham-Delègue, 1991). There is evidence that bees have greater success in learning during August through November compared to March through July. However, the acquisition curves for these studies show that the bees learned the CS-US pairing without a problem, which indicates that the season was not an issue.
The acquisition curves show that the bees were able to learn the pairings between the CS and the US, but they did not appear to have learned which US to expect, based on the presentation of the CS. Researchers have found conflicting results on the extent of the honey bees' learning abilities (Craig & Abramson, 2015), which indicates that we do not completely understand the cognitive capabilities of bees. Some issues in past literature are (1) a failure to properly control for variables, (2) researchers having issues with selecting the proper instrumentation, and (3) issues with not being able to replicate studies due to lack of information (Craig & Abramson, 2015). These issues have resulted in an inability to compare study results in some cases (Craig & Abramson, 2015).