Anim Cogn (2015) 18:1143–1154 DOI 10.1007/s10071-015-0887-8

ORIGINAL PAPER

Conditional discrimination and response chains by worker bumblebees (Bombus impatiens Cresson, Hymenoptera: Apidae) Hamida B. Mirwan1 • Peter G. Kevan1

Received: 31 July 2014 / Revised: 8 June 2015 / Accepted: 15 June 2015 / Published online: 7 July 2015 Ó Springer-Verlag Berlin Heidelberg 2015

Abstract We trained worker bumblebees to discriminate arrays of artificial nectaries (one, two, and three microcentrifuge tubes inserted into artificial flowers) from which they could forage in association with their location in a three-compartmental maze. Additionally, we challenged bees to learn to accomplish three different tasks in a fixed sequence during foraging. To enter the main three-compartmented foraging arena, they had first to slide open doors in an entry box to be able to proceed to an artificial flower patch in the main arena where they had to lift covers to the artificial nectaries from which they then fed. Then, the bees had to return to the entrance way to their hive, but to actually enter, were challenged to rotate a vertically oriented disc to expose the entry hole. The bees were adept at associating the array of nectaries with their position in the compartmental maze (one nectary in compartment one, two in two, and three in three), taking about six trials to arrive at almost error-free foraging. Over all it took the bees three days of shaping to become more or less error free at the multi-step suite of sequential task performances. Thus, they had learned where they were in the chain sequence, which array and in which compartment was rewarding, how to get to the rewarding array in the appropriate compartment, and finally how to return as directly as possible to their hive entrance, open the entrance, and re-enter the hive. Our experiments were not designed to determine the specific nature of the cues the bees used, but our results strongly suggest that the tested

& Peter G. Kevan [email protected] 1

School of Environmental Sciences and The Canadian Pollination Initiative, University of Guelph, Guelph, ON N1G 2W1, Canada

bees developed a sense of subgoals that needed to be achieved by recognizing the array of elements in a pattern and possibly chain learning in order to achieve the ultimate goal of successfully foraging and returning to their colony. Our results also indicate that the bees had organized their learning by a hierarchy as evidenced by their proceeding to completion of the ultimate goal without reversing their foraging paths so as to return to the colony without food. Keywords Bees  Foraging  Fixed sequence  Pattern recognizing  Response chain

Introduction In solving several problems during a period of activity, animals may use various strategies. In general, subjects are able to attain their goals by having acquired, through experience, the ability to respond to different objects or events as they are encountered. Upon encountering any of the particular objects, they respond by conditional discrimination and learned responses so as to continue. The concept of conditional discrimination ranges from being manifested by an animal’s responding in one way to one stimulus and a different way to another (Mackintosh 1974: 543) to an animal’s response to a particular stimulus that depends on context but is conditional on the entire stimulus configuration rather than a particular element of it (Mostofsky 1965: 284–330). In essence, for conditional discrimination to be exhibited, the test animal responds as follows: if the condition is ‘‘A’’, then the rewarding/reinforcing response is ‘‘AR’’, but if ‘‘B’’, then ‘‘BR’’, and so on. Lashlev (1938), in his pioneering studies, conditioned rats to differentiate between upright and inverted triangles targets on black or striated backgrounds to access the

123

1144

reinforcer, and his results were replicated by North et al. (1958). Smith (1951) studied conditioned responses on rats responding in a T-maze in which their passage through the tube (with white or back walls) to the choice at the T-junction was conditioned by the colour of the tube (turn left if black but right if white). Rats have since been used by many researchers to investigate conditional discrimination in a huge variety of contexts (see Skinner 1938, 1953; Pearce 2008; Domjan 2015). Conditional discrimination has been demonstrated also in rabbits (Saavedra 1975), goldfish (Bitterman 1984), and pigeons (Carter and Werner 1978; Schrier and Thompson 1980; Thomas et al. 1988) trained with combinations of visual and auditory stimuli. Pigeons have also been conditioned to specific combinations of colour and form (Born et al. 1969). Although there is an extensive literature on conditional discrimination learning by vertebrates, few studies demonstrate conditional discrimination by invertebrates (e.g. cephalopods (Grasso and Basil 2009) and honeybees (below)). Conditional discrimination has been observed in honeybees (Apis mellifera) (Couvillon and Biiterman 1988, 1989, 1991; Funayama et al. 1995), which were trained on two different conditional problems that required them to discriminate between two differently coloured objects on the basis of an odour or that required them to discriminate between two differently scented targets on the basis of same coloured objects. Giurfa and Nu´n˜ez (1992, 1993) showed that honeybees discriminated between pheromonemarked artificial flowers and non-marked, or with scent marks experimentally eliminated. In our experiments, the condition was the bees’ location in a three-compartment arena and the rewarding/reinforcing response was to discriminate between three kinds of artificial flowers, only one of which was rewarding depending on compartment. As early as 1890 (James 1890) (e.g. vol I: Chapter 14), James noted that the appreciation of sequential events must be important in the behaviours of animals, but as Weisman et al. (1980) discussed in their introductory review, little attention was paid to that aspect of learning and cognition. Response chain learning, also called serial recognition (Pearce 2008: 253–257) and linking (Taylor et al. 2010), involves the subject acquiring skills to perform a series of tasks in order, so that one correct response provides the cue for next and it is the last correct response that produces a reinforcer (Skinner 1938, 1953). Chained responses may produce, or alter, some of the variables which control other responses (Skinner 1953) (as in the studies of Balleine et al. (1995) with rats that pressed a lever and then pulled a chain to obtain the reinforcer), but that situation may have been complicated by ‘‘chunking’’ (Terrace 1987, 1991) by which combinations of stimuli

123

Anim Cogn (2015) 18:1143–1154

presented simultaneously (simultaneous chaining) make for improved cognition versus single stimuli alone. Conditional discrimination is shown when an animal responds differently to the same stimulus presented in differing contexts (Mostofsky 1965), and response chaining is shown when an animal responds to a fixed sequence of objects or events in order to gain a primary reinforcer (Pearce 2008). However, sequence learning (or chaining) also involves the subjects’ responding by conditional discrimination and learned responses so as to continue its activity but is shown by the subject’s having acquired, through experience, the ability to recognize different objects or events as they are encountered one after another in a particular, fixed, order (Dehaene 1999). Sun and Giles (2001) suggest that sequence learning could be the most important and prevalent kind of learning. The subject must react by experience to: (1) sequence prediction, (2) sequence generation, (3) sequence recognition, and (4) sequential decision-making (Sun 2001; Sun and Giles 2001). Sequence learning might be dependent on an animal’s ability to organize learned behaviours hierarchically into behavioural chains with goals and subgoals (Byrne and Byrne 1993; Byrne and Russon 1998). We consider that an animal’s tackling of those four interconnected problems requires that the animal have some understanding of where it is in the sequence, i.e. have some means of assessing its accomplishments as it progresses, and what it must do at each point in the sequence. In two different experiments, both requiring learning an array of complex tasks with delayed rewards and different rewards, we conditioned worker bumblebees in Experiment 1 to discriminate the pattern (by possibly counting, subitizing, or otherwise enumerating, see Pearce 2008) one, two, or three artificial nectaries presented in arrays in three different compartments (thus recognizing a pattern of objects), and also knowing in which compartment in sequence (the conditional context) they were during the same task. Then, in Experiment 2, the bees were challenged (trained) to solve an array of complex manipulative problems with goals and subgoals (conditioned reinforcers). Ultimately, their goal (primary reinforcer) was to exit the hive, forage, and return while having to react correctly to a sequence of challenges. The subgoals were first, to open a sliding door in a small entry box to gain entry to the main three-compartmental foraging arena, the second was to push up occluding caps over the artificial flowers to obtain the syrup reward reinforcer, and then third to rotate a disc to re-enter the hive (which, in this case, was the ultimate—called primary—reinforcer). Our experiments were not designed to determine the specific nature of the cues the bees used.

Anim Cogn (2015) 18:1143–1154

Materials and methods General methods Foragers of bumblebees (Bombus impatiens Cresson: Hymenoptera: Apidae) from queen-right colonies of 30–40 workers/colony (supplied by BioBest Biological Systems Canada, Leamington, Ontario) were used in the experiments. When not being tested, colonies were provided with a constant supply of pollen and sugar syrup. Pollen was collected from Western honeybee (Apis mellifera) hives by use of an OAC pollen trap at the apiculture research facility at the University of Guelph: the harvested pollen pellets were frozen for storage, but thawed and warmed to ambient air temperature to be fed to the bees used in our research. The syrup was 50 % w:w sucrose in double-distilled water made fresh daily. Four different colonies were used in the experiments. Experiments were conducted in indoor screened flight cages (2.15 m long 9 1.20 m wide 9 1.80 m tall) with grey floors. A moveable screen on one side of each cage allowed access. One bumblebee colony was connected to a small, outer cage (30 9 23 9 20 cm) that served as a holding area. The holding area was attached to the main flight cage (testing arena) by gated, wire-mesh tunnels that allowed control of the bees’ entry to the flight cage and the maze. During experiments, bees exiting the hive could take only one route through the holding area to the testing arena (the gate on the diagonal route was kept closed). The gates between the holding area and the testing arena were manipulated as needed to allow single bees to enter the testing arena. Once in the testing arena, bees had to access a feeding area that was located 165 cm from entrance and exit points. The feeding area consisted of a green Styrofoam plate 45 9 35 9 5 cm with eight holes that held centrifuge tubes (1.5 ml). The tubes, hidden from the bees, were filled with 50 % sucrose solution (w:w made with deionized water fresh daily) (syrup) as the reinforcer. The amount of syrup was not controlled, but was replenished as soon as it was exhausted. After foraging in the feeding area, bees were allowed to return to the hive via the diagonal route. The first step was to allow bees to forage for syrup in the feeding area, at the far end of the testing arena from the colony’s access, where eight microcentrifuge tubes mounted in a Styrofoam base. Once the bees were accustomed to foraging at those feeders for a week to ten days, they were marked individually on their thoracic dorsal surfaces with uniquely numbered and coloured tags (Opalith Pla¨ttchen, Christian Graze KG, Germany), and then challenged with learning tasks as described for each experiment (below).

1145

Experiment 1: conditional discrimination: testing bumblebees’ abilities to recognize elements in an array depending on placement in three interconnected compartments Within a three-compartmented arena in which the bees learned to forage and return to their hive by an increasingly complex and convoluted path, we conditioned the bees’ abilities to discriminate three types of flowers: the first with one nectary tube, the second with two nectary tubes and the third with three nectary tubes. Each tube had its own entrance. As conditioning progressed, the bees were trained to recognize that it was the array of one nectary that was rewarding in compartment one, two nectaries in compartment two, and three nectaries in compartment three. Artificial flowers Three types of artificial flowers (with one, two, and three nectaries) were used (Fig. 1a) Experimental set-up We used two cages (described above) each with one bumblebee hive attached (Fig. 1b). Two partitions were used to make a three-compartment maze in each cage. The partition closest to the point at which the bees entered was a white opaque plastic sheet. The second partition was cardboard. To travel between the compartments (C1, C2, and C3), the bees had to fly through a square hole (15 cm2) cut through each partition. The hole closest to the point at which the bees entered the cage was 55 cm above the floor and 5 cm from the right wall. The second hole (in the cardboard) was 100 cm above the floor and 55 cm from the right wall. Thus, the subject bees had to fly a convoluted trajectory through the maze from entering the cage to reaching the back wall of the last (third) compartment (Fig. 1b) and back. At the back of each compartment, an array (patch) of 12 artificial flowers was displayed at an angle of 110° (the angle of presentation was enough to stop the sugar syrup within the tubes from running out). Each patch included four flowers from each type (one, two, or three nectary tubes) arranged as in Fig. 1a flower patch A or flower patch B, with one being the inverted version of the other. We placed the flowers with two and three nectary tubes in two orientations as shown in Fig. 1a. To decide on the initial arrangement of the three flower types in four rows each with all three flower types, we arbitrarily assigned the positions of the flowers in the order shown in Fig. 1a, patch A. This way of arranging the flowers in patches was chosen for practical reasons (considering the

123

1146

Fig. 1 Designs and diagrams for materials and apparatus for Experiment 1. a Illustrates flower patches A and B. b Illustrates the experimental set-up with hive, holding area, flight cage testing arena divided into a three-compartment maze. The compartments were connected by holes cut into the dividers. An array of artificial flowers was mounted on a Styrofoam base in each compartment. Mesh tube routes with gates allowed the bees to enter and exit the flight cage. The bees, in training or as trained, exited from the hive and could take only one route through the holding area to the testing arena in the main flight cage. The trajectories of the bees illustrated show outgoing foragers stopping on the floral array in the first compartment (rewarded during training but not when assessing array of artificial flowers and accomplishments were being tested), going on to the second and stopping (again rewarded during training but not during

123

Anim Cogn (2015) 18:1143–1154

testing), and then proceeding to the third (always rewarded). On returning to their hive, the bees’ trajectories did not have stops at the floral arrays. The bees entering the flight cage were not allowed to use the diagonal mesh tube because its gate was kept closed. The gates after the holding were opened and closed to allow only single bees to enter the testing arena during testing. The bees returned to their hive from the testing area via the diagonal mesh tube route, the gate of which was opened as necessary. Type a Flower N1: a simple flower was made of 7 cm of blue plastic disc. A single 0.5-ml centrifuge tube (nectary) was attached to the disc at its centre. Type b Flower N2: same as type (a) with two centrifuge tubes (nectaries) attached to the disc across the centre. Type c Flower N3: as (a) with three centrifuge tubes (nectaries) attached to the disc at the centre arranged in a triangle

Anim Cogn (2015) 18:1143–1154

number of total possibilities) while providing patches without obvious potential for the bees to recognize patterns and positions of the flowers. Experimental procedure We started by training the bees to forage from their hives into the three compartments with flowers and their nectaries filled to the brim with 50 % fresh syrup. On the first day of training, we used fully loaded flowers in each compartment. The bees were allowed to forage freely. The bees emptied the flowers in the first compartment and so had to find their way into the second and forage there. Once the flowers in compartment 2 were emptied, the bees had to progress to the third. Thus, the bees had become familiar with the flower patches, the nectaries in each flower, and the tri-compartmental arena in which to forage. On Days 2 and 3, in compartment 1, those flowers with a single nectary were filled with syrup, but the flowers with two and three nectaries were filled with water. At the same time, the flowers in compartment 2 were filled, those with two nectaries with syrup and those with one and three nectaries with water. In compartment 3, we filled the flowers so that only those with three nectaries had syrup and the rest had water. The bees were then allowed to forage freely. Thus, the bees had to become familiar with the flower patches, and the numbers or pattern of rewarding nectaries in each flower in association with the compartmental sequence. All rewarding flowers were refilled after becoming depleted. After the bees had learned that the rewarding flowers differed according to number or pattern of nectaries and in which compartment they were located, the experiment was started on Day 4. After the three days of training, we chose 24 bees (of 30 that started) that were successful in learning to travel throughout the cage and associate the number or pattern of nectaries with the rewarding flowers in each compartment for further testing. For further testing, we removed the syrup reward from all the flowers in compartments 1 and 2 and filled them with water. Only those flowers with three nectaries in compartment 3 remained rewarding. After each bee had foraged successfully, the flowers that had been visited were replaced with clean ones. The flower patches were rotated 180o (patch A vs. patch B) to reduce the possibility of the bees’ learning floral positions after individual foraging bouts by all the bees. Thus, each bee had to enter the cage, test the flowers and the nectaries in compartment 1, then finding only water provided, had to repeat the exercise in rewardless compartment 2, and then had to enter compartment 3 to obtain syrup from only those flowers with three nectaries. During the experiment, we recorded the bees’ choices of compartments and flowers for each of 10 foraging bouts (trials).

1147

Experiment 2: response chains: bumblebees manipulating multiple obstacles In this experiment, we investigated bumblebees’ abilities to operate different obstacles (Fig. 2a–c) in serial order by presenting them to the subjects in order: first, sliding doors (squares); second, lifting an obstacle to access the artificial flowers; and finally, rotating a disc to allow them to return to the hive. Materials Sliding doors (Fig. 2a): A black (interior and exterior) box (30 9 20 9 15 cm) with a front of transparent plastic (for viewing) was placed inside the testing cage and connected by mesh tube to the hive. Opposite the transparent wall the box had four holes, each one covered with a square sliding door (4 cm2) of pink ‘‘Eva’’ foam sheet which, when opened, allowed the bees to exit the box and enter the cage. These doors could be opened by being slid along horizontal tracks above and below. Eva foam (Fuzhou Vlin Plastic Products Co., Ltd. Fujian, China) is made of metallocene polyolefin (POE) elastomer blended with ethylene vinyl acetate (EVA). Artificial flowers (Fig. 2b): Eight artificial flowers were made with centrifuge tubes 1.5 ml, with the caps removed. The tubes were inserted through the edge of blue acetate discs (7 cm diameter). Each of the caps (1.5 cm) was attached to a 4.5-cm-long one-sided cotton swab, and the assembly anchored with a pin through the cotton so that the cap occluded the entry to the centrifuge tube. To gain access to the reinforcer syrup, the subjects had to lift the caps. A pin on either side of the rod of the cotton swap prevented lateral movement. The arrays of flowers were mounted on a 45 9 35 9 5 cm rectangular Styrofoam plate. Rotatable disc (Fig. 2c): A white Eva foam disc (12 cm) with one edge cut to make a truncated sector (wedge) with its maximum dimension of quarter of the circumference was equipped with a small (3 9 7 mm) black Eva foam tag attached near the short, radial, edge of the removed wedge. The tag provided subject bees with a visible signal and grip to rotate the disc. This disc was placed vertically over the circular 1.5-cm hole providing exit from the testing cage into the mesh tube leading back to the hive. Experimental set-up As in Experiment 1, two cages with attached bumblebee hives were used (Fig. 2d). The black box, with a transparent plastic front (for observation), was placed 15 cm from cage’s solid wall through which the mesh tube from the hive was inserted,

123

1148 Fig. 2 Designs of materials and apparatus used for Experiment 2 in which bumble bees solving the problems posed by sequential obstacles. a Front view of the black box. b Design of the artificial flower requiring the bees to push up the occluding cap. c Eva foam rotatable white disc mounted on the testing arena wall to block direct entry by the bees into their hive. d Diagram of the experimental set-up with hive, holding area, black box with sliding doors within the flight cage testing arena, and artificial flower patch, mesh tube routes with gates by which the bees were allowed to enter and exit the flight cage

Anim Cogn (2015) 18:1143–1154

(a) Hole

(b) Eva foam sliding square

Black pin

Black Eva foam tag

Blue disc

Entry

4.5 cm one side cotton

White disc Entry hole

Tracks

Black box

Centrifuge tube

Cage wall

(d)

and close (5 cm) to the front screen wall of the cage. The square doors inside the box could to be slid to only the right to allow egress (first reinforcer) to the main cage and to the artificial flowers. The patch of eight artificial flowers (Fig. 2b) was presented at the opposite end of the cage from the hive. On the artificial flowers, subject bees had to lift the movable caps to access the second reinforcer of sugar syrup. Finally, a rotatable white Eva foam disc hid the entrance of the mesh tube leading from the main cage to the hive. Subject bees have to rotate the disc upwards to uncover the entrance hole and gain access to their nest mates (the third reinforcer). Experimental procedure After the bees learned to forage for 7 days in the threecompartment main cage without the door box, with simple artificial flowers (no caps over the nectary) at the cage’s end, and no rotating disc at the hive end of the cage, the bees were individually marked (as described above) and further training began. On the next day (Day 1), the black box was attached with all four holes open and the front covered with a black sheet to allow light to enter only from the four holes. Thus, the bees

123

(c) Centrifuge tube cap

Pins

demonstrated that they had learned their way through. In the middle of the same day, the black sheet was removed. On Day 2, the doors were slid to the left to cover half of each hole. Then, in the middle of the day, the doors were slid to cover almost the entire hole. The subject bees had to learn that the holes were hidden by the doors and how to slide them to the right to pass through. During Day 2, the bees foraged from the simple artificial flowers. The 14 individuals that failed to slide the doors were removed from the box and eliminated from further tests and data analysis. On Day 3, without the door-box doors being closed, the bees were trained to forage from the complex artificial flowers with caps that needed lifting (Fig. 2b). On Day 4, when the last task for the bees was presented with the doorbox doors open and with simple artificial flowers, they had to learn how to rotate the white Eva foam disc (Fig. 2c), to access their hive after foraging. The disc was presented first so that it did not occlude the hole, but was then lowered to cover half of the hole, and finally to cover almost all the hole. At this stage, it was not dropped to completely occlude the hole’s location (as in the final trials). The three individuals that could not rotate the disc were excluded from further tests and data analysis.

Anim Cogn (2015) 18:1143–1154

After the bees learned each of those independent tasks, on Day 5, they were tested with all three tasks: (1) doorbox doors fully to the left covering the holes and requiring the sliding open of one door, (2) foraging from the complex artificial flowers with movable caps in place requiring lifting to access the syrup, and (3) to rotate the white disc, now completely covering the hive entry tube, to enter to their hive. The time each individual bee took to finish each task and the total obstacle course was recorded for 10 trials for each bee. Limitations of methods In Experiment 1, we did not provide randomized arrays of one, two, and three nectaries that would be required to test the numerosity hypothesis, i.e. that the bees were counting. In Experiment 2, we used a fixed sequence of tasks the bees had to perform and that would need to have been varied to test the chaining (sequence learning) hypothesis. Both experiments, as complex as they are, suggest fertile ground for more comprehensive studies. Further, we did not combine the challenges of experiments 1 and 2 which would have involved (a) the three-compartmental with single-, double-, and triple-nectary artificial floral arrays and (b) entrance box with sliding doors, manipulation of the caps occluding the nectaries, and the rotating entrance to the hive. Data analysis Statistical analyses (by Systat Software Inc., version 12.2.43f) tested our behavioural hypotheses that worker bumblebees can learn to accomplish suites of tasks by conditional and chained responses by comparing the mean durations taken by the bees, after entering the test arenas, to perform the tasks with which they were presented. Oneway ANOVAs for repeated measures were used to compare bees’ speeds of performance across trials and across tasks. Experiment 1: Conditional discrimination: Recognizing the array of nectaries (tubes in artificial flowers, see Fig. 1b). One-way completely randomized ANOVA was used to compare bees’ choices of the correct flower (number of nectaries in the respective compartment) on each patch. Experiment 2: Response chains: Manipulation of multiple obstacles (Fig. 2). To compare between the bees and their individual durations to finish each obstacle and the total course, an one-way repeated ANOVA measurement was used. To identify all pair-wise multiple differences, Tukey’s test was used. The learning rates were estimated by two statistics. We subtracted the mean time for completion of the task from

1149

the time it took on the first encounter for each of the 10 bees. We also used the power law of practice (Newell and Rosenbloom 1981; Delaney et al. 1998; Ritter and Schooler 2002; but see Heathcote et al. 2000) as a generally wellaccepted mathematical model for the learning curves derived from our results and used the exponent as a measure of learning rate. We used Sigma Plot (Systat Software Inc., version 12.2.43f) to generate the least-squares best fits of our results (R2) to the derived equations (y = Axz where A is the scaling factor (the higher the value, the more timeconsuming the task is to perform in the first trial) and z is the exponent (negative for our data), indicating the proportionate decrease in time to complete the task from trial x to trial x ? 1 (higher values of jzj indicate greater rates of learning)). We used these fitted curves to judge visually the point at which asymptotic performance was reached in each task or experiment.

Results Conditional discrimination: recognizing the number of nectaries (Experiment 1) Our results showed that bumble bees have the capability of recognizing the pattern or number of elements in an array of one, two, and three artificial nectaries. The subjects learnt to choose the rewarding flowers of each patch in direct accordance with the compartment they were in. The mean value of flower choice of each patch corresponding with previous experience was great enough to conclude that their choice was not random and their previous experience affected that selection. Results from the bees’ visits to flower patches A and B could not be separated because the patches are the same except for vertical orientation and that was reversed after each foraging bout (as explained by the experimental design) In compartment 1, the bees chose one nectary more than they did two or three nectaries (F2,9 = 46.42; P \ 0.0001). In comparing the performances (choosing an artificial flower with a single nectary) of all the individual bees in compartment 1, inter-individual difference in learning performance between tested bees was not significant (F2,9 = 2.78; P = 0.11) (Fig. 3a). In compartment 2, the bees chose flowers with two nectaries more than they did one or three nectaries (F2,9 = 124; P \ 0.0001). In comparing the performances (choosing an artificial flower with two nectaries) of all the individual bees in compartment 2, inter-individual difference in learning performance between tested bees was not significant (F2,9 = 1.25; P = 0.32) (Fig. 3b).

123

1150

Anim Cogn (2015) 18:1143–1154

(b)

2

3

Flowers with

s ct ar ie

Flowers with

Compartment 3

(c) 30 25 20 15 10 5

rie

rie

ta ec N 3

2

N

ec

ta

ta ec N 1

s

s

0

ry

Mean number of visitation

Ne

1

N

ec ta rie s 2

1

N

N

ec ta rie s

0

3

5

s

10

ct ar ie

15

Ne

20

Ne

25

Compartment 2

50 45 40 35 30 25 20 15 10 5 0

ct ar y

Mean number of visitation

Compartment 1

30

ec ta ry

Mean number of visitation

(a)

Flowers with Fig. 3 Mean number of bees (N = 24) landing on trained flower type (the flower rewarded depending on the number of nectaries (tubes)) of three of similar flower patches for 10 trials. a The mean number of bees landing on the three types of flowers on first patch where only arrays of one nectary were rewarding and the others were not, b the

mean number of bees landing on the three types of flowers on second patch where only arrays of two nectaries were rewarding, and c the mean number of bees landing on the three types of flowers on third patch where only arrays with three nectaries were rewarding

In compartment 3, the bees chose three nectaries more than they did one or two nectaries (F2,9 = 534; P \ 0.0001). In comparing the performances (choosing an artificial flower with three nectaries) of all the individual bees in compartment 3, inter-individual difference in learning performance between tested bees was not significant (F2,9 = 1.15; P = 0.37) (Fig. 3c). Not only had the bees learned to associate the array of nectaries with the compartment in which they were foraging, they had learned that compartments 1 and 2 were rewardless, even though they had single and double nectaries in the arrays of artificial flowers presented. Eight of the 24 bees in their final trial learned to traverse compartments 1 and 2 without stopping. Some bees (7), even in their final trial, did stop in compartment 1 and probe singlenectary artificial flowers with only water in them, but then flew on. Some bees (11) in their final trial did not stop in compartment 1 and then probed double nectary artificial flowers with only water in them in compartment 2, but then

flew on to forage correctly in compartment 3. Some bees (5) in their final trial did stop in compartment 1 and probe single-nectary artificial flowers with only water in them, and then stopped in compartment 2 and probed double nectary artificial flowers with only water in them, but finally flew on to forage correctly in compartment 3.

123

Experiment 2: response chains: manipulating multiple obstacles Our results show that worker bumblebees can learn to operate three different and complex tasks in serial order. The mean durations amongst the bees to solve the first obstacle, sliding doors, were not significantly different (F9,9 = 1.9; P = 0.18); however, as expected, there was a statistically significant difference between trials as the bees learned (F9,9 = 20; P \ 0.0001) (Fig. 4a). With the second obstacle, lifting caps occluding access to the nectary of an artificial flower, again the bees did not

Anim Cogn (2015) 18:1143–1154

1151

differ in manipulation time amongst themselves (F9,9 = 1.5; P = 0.28), but there was a statistically significant difference between trials as the bees learned (F9,9 = 4; P = 0.026) (Fig. 4b). For the final obstacle, rotatable disc, our results show the same pattern as with the other obstacles of no significant difference between the bees for their manipulation times (F9,9 = 1.2; P = 0.40) but significant differences between trials (F9,9 = 33; P \ 0.0001) (Fig. 4c). In assessing the bees’ performances in completing the whole ‘‘obstacle course’’, we also found no significant

difference between the bees for their manipulation times (F9,9 = 2.0; P = 0.16) but significant differences between trials (F9,9 = 37.0; P \ 0.0001) (Fig. 4c). As expected, the different tasks took different times to accomplish and were learned at different rates (Table 1, Fig. 4). Nevertheless, in the final trials, the bees took almost the same amounts of time to manipulate each obstacle (Fig. 4). As our experiments proceeded, we noted that the bees employed similar techniques to slide open the door, lift the cap, and rotate the disc. To slide doors to the right, they 50

Mean time/ Sec

120 100 80 60

y = 116.68x-1.165 R² = 0.9322

40

Mean time/Sec

140

40 30

y = 21.85x-0.445 R² = 0.7091

20 10

20 0

0 1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

8

9

10

Trial

Trial

(b) Lifting caps

(a) Sliding doors 70

Mean time/Sec

60 50 40

y = 38.222x-0.955

30

R² = 0.9296

20 10 0

1

2

3

4

5

6

7

8

9

10

Trial

(c) Rotating disc Fig. 4 Mean (±SE) of the durations (s) taken by the bees (N = 10), to manipulate each obstacle during 10 trials and the power function best fits to the curves of results in Experiment 2. a Improving performance in sliding doors of the Box (Fig. 2a) to the right to enter the flight cage and forage, b improving performance of pushing up caps occluding the entrance to the syrup (Fig. 2b) to start feeding, and c improving performance in rotating the disc (Fig. 2c) to get access to

their hive. Comparisons of the times taken to solve each of the three tasks (a, b, and c) indicate that the times taken for the first trial for each of 10 bees for each obstacle differed significantly (F2,9 = 12; P = 0.0002), but by the 10th trial for each of 10 bees for each obstacle, the times taken were statistically the same (F2,9 = 2; P = 0.19)

Table 1 Learning rates of worker bumblebees for the different obstacles to be manipulated as calculated by the difference in time between first and 10th encounters, and by the exponent of the power function for the learning curve over 10 trials Obstacle type Lifting caps

Time to accomplish task on first encounter

Time to accomplish task on 10th trial

28.1

6.8

Rotating discs

53.5

4.6

Sliding doors

112.3

7.3

Learning difference (first-last)

Power function exponent

21.3

-0.44

48.9

-0.95

105

-1.16

Time is measured in seconds

123

1152

used their right middle (mesothoracic) legs. To lift caps, they inserted their proboscides beneath the cap and then lifted with their proboscides and mesothoracic legs. To rotate the disc preventing immediate entry to the hive, they pushed with their frons and both mesothoracic legs.

Discussion The primary question we addressed in both experiments was ‘‘can bumblebees learn to perform a sequence of complex tasks using conditional discrimination and a fixed order, i.e. as a response chain?’’ We consider that an animal’s tackling of a sequence of tasks requires that it have some understanding of where in the sequence it is (the conditional context), i.e. it should have some means of counting its accomplishments as it progresses. In our first experiment on conditional discrimination, we trained the bees to associate the pattern (or possibly number) of nectaries (one, two, and three) with the position that the bees found themselves in navigating a three-compartment maze. The subject bees were able to make that association so that they learned to forage on artificial flowers with one nectary (but not those with two or three) in the first compartment, two nectaries (but not those with one or three) in the second, and, in the most complex challenge, they foraged at artificial flowers with three nectaries (but not those with one or two nectaries) in the third compartment of the maze. Others have shown counting abilities in a wide variety of animals (Pearce 2008; Pahl et al. 2013), including insects (landmarks and dots by honeybees (Chittka and Geiger 1995; Dacke and Srinivasan 2008; Gross et al. 2009) and nectaries (Bar-Shai et al. 2011a, b)) and in cicadas, time intervals (Karban et al. 2000). Also, visual decision-making based on number recognition has been invoked in honeybees by Leppik (1953) in experiments where pattern recognition may have been more important and by Gross et al. (2009) who showed that honeybees could learn, in a Y-maze, to distinguish 2 versus 3 versus 4 randomly placed dots, but were unable to distinguish between 4 and more dots. Experiments on conditional discrimination in honeybees (Apis mellifera) (Couvillon and Biiterman 1988, 1989, 1991; Funayama et al. 1995) involved two problems that required discrimination between two differently coloured objects on the basis of odour or that required discrimination between two differently scented targets on the basis colour. Our results indicate that workers of B. impatiens learned to recognize the arrays of nectaries, possibly by pattern recognition or but also possibly by subitizing. Furthermore, the bees must have known which compartment of the maze they had reached. Having found that worker bumblebees of B. impatiens could discriminate the patterns of objects (as shown for B.

123

Anim Cogn (2015) 18:1143–1154

terrestris by Bar-Shai et al. 2011a, b), we challenged them in a second experiment to learn to accomplish complex tasks in a fixed sequence. Although the experiment does not demonstrate chaining (in the strict sense), our results correspond to sequence recognition and sequential decision-making through several actions: goal-oriented, trajectory-oriented, and reinforcement-maximizing, all of them lead to the goal (in our experiment, obtaining sugar syrup and returning home) at the end (Sun 2001). Our results show that the bees learnt a series of complex tasks (1) sliding doors to gain entry to their foraging arena, (2) orienting to an array of eight artificial flowers and lifting the caps covering the nectaries to obtain the hidden sugar syrup, and (3) returning to the entry point to their hive where they had to rotate a disc to expose the tunnel to the hive. Thus, they had accomplished solving a multi-step set of problems in a fixed order. Previous studies have shown that many kinds of animals including insects can navigate or overcome barriers, e.g. ants (Chameron et al. 2011), honeybees (review in Collett et al. 1993), and bumblebees (Chittka and Thomson 1996; Mirwan and Kevan 2015). Menzel (1990) suggested that honeybees do have, to some extent, an internal representation of sequential tasks. Martin (1965) showed that honeybees were able to react to a series of four scents, of which only one sequence of odours leads to the reward. Gegear and Laverty’s (1995, 1998) research indicates intertask interference constrains solution of several tasks presented simultaneously, not in sequence. Our experiment is different because it involves three rather different obstacles, each presented in sequence, that need manipulation skills (sliding, lifting, and rotating) coupled with navigation and numerical pattern recognition (perhaps with counting). Our experiments and results are different from associative chaining or linking theory (e.g. in which each response becomes the stimulus for the next) (Spiegel and McLaren 2006) and numeracy because we have linked the relationship between recognizing arrays of objects while knowing where in a sequence of tasks an animal might be. That combination relates to problem-solving, navigation and orientation, and cognition. Moreover, we have presented to bumblebee workers a fixed series of tasks that may demonstrate that they organize learned behaviours hierarchically into behavioural chains with goals and subgoals (Byrne and Byrne 1993; Byrne and Russon 1998). After passing through the sliding door box, the bees had a choice of returning to their hive or proceeding to the floral array, lifting the occluding caps to forage before returning home. Clearly, their priority was to proceed and forage. The second and more complex type of response chains includes problems in which tasks may include barriers which require operation by the test subjects, and all

Anim Cogn (2015) 18:1143–1154

obstacles must successfully negotiated in order to complete the task, as shown for pigeons (Columba livia) (Kohler 1925; Birch 1945; Epstein et al. 1984; Epstein 1987), New Caledonian crows (Corvus moneduloides) (Taylor et al. 2010), chimpanzees (Pan troglodytes) (Do¨hl 1968), and bumblebees (Bombus impatiens) (Mirwan and Kevan 2014). The size and type of obstacles vary according to the tasks, but response chaining coupled with problem-solving has not been tested in invertebrates until now, as far as we know. Although our results suggest chaining (sequence learning), we cannot firmly invoke it as the learning paradigm of our test bees because we did not vary, or provide choice, in the sequence of tasks. We are sure that such experiments would yield interesting results. Pattern recognition, counting, and solving complex problems by response chaining are part of the foraging strategies of bees. Our results show that the capacity of worker bumblebees to solve different complex problems through conditional discrimination and in a particular order involves their knowing where in a sequence of tasks they are (possibly counting and prioritizing the tasks) and being able to recognize a positional or numerical array of objects as they perform some tasks. Gegear and Laverty (1998) found that the capacities of bees to perform different complex tasks when presented simultaneously were constrained, but our experiments show that such constraints are not as severe when different complex tasks are presented sequentially. Floral fidelity/constancy may be constrained by bees being unable to perform several complex manipulative tasks within a mixed patch of flowers, but are likely less constrained when moving between patches of different flowers, even on single foraging trips. Their abilities to perform different complex tasks in sequence may be an important part of bees’ capacities to sample different flowers as they forage and explain how they ‘‘major’’ and ‘‘minor’’ (Heinrich 1979) in tracking changing resource availabilities over time and space. Acknowledgments We thank the Canadian Pollination Initiative (NSERC-CANPOLIN for which this is publication No. 135) for funding some of this research reported. HM thanks the Libyan Ministry of Education and Canadian Bureau for International Students for scholarships received. BioBest Biological Systems, Leamington, Ontario, kindly provided colonies of the test subjects. We are especially grateful to Drs T. Woodcock and Francesco Leri, CANPOLIN, and Psychology Department, respectively, of the University of Guelph for critical reviews and help in preparing this paper. The studies reported herein comply with Canadian ethical standards, and those of the University of Guelph, for research and treatment of experimental animals Compliance with Ehical Standards Conflict of interest There is no conflict of interest with any of the sponsors of the research reported herein.

1153

References Balleine BW, Garner C, Gonzalez F, Dickinson A (1995) Motivational control of heterogeneous instrumental chains. J Exp Psychol: Anim Behav Proc 21:203–217 Bar-Shai N, Keasar T, Shmida A (2011a) How do solitary bees forage in patches with a fixed number of food items? Anim Behav 82:1367–1372 Bar-Shai N, Keasar T, Shmida A (2011b) The use of numerical information by bees in foraging tasks. Behav Ecol 22:317–325 Birch H (1945) The relation of previous experience to insightful problem-solving. J Comp Psychol 38:367–383. doi:10.1037/ h0056104 Bitterman ME (1984) Migration and learning in fishes. In: McCleave ID, Arnold GP, Dodson II, Neill WH (eds) Mechanisms of migration in fishes. Plenum, New York, pp 397–420 Born DG, Snow ME, Herbert EW (1969) Conditional discrimination learning in the pigeon. J Exp Anal Behav 12:119–125 Byrne R, Byrne A (1993) Complex leaf-gathering skills of mountain gorillas (Gorilla g. berengei): variability and standardisation. Am J Primatol 31:521–546 Byrne R, Russon A (1998) Learning by imitation: a hierarchical approach. Behav Brain Sci 21:667–721. doi:10.1017/ S0140525X98001745 Carter DE, Werner TJ (1978) Complex learning and information processing by pigeons: a critical analysis. J Exp Anal Behav 29:565–601 Chameron S, Schatz B, Pastergue-Ruiz I, Beugnon G, Collett TS (1998) The learning of a sequence of visual patterns by the ant Cataglyphis cursor. Proc R Soc Lond B 265:2309–2313 Chittka L, Geiger K (1995) Can honeybees count landmarks? Anim Behav 49:159–164 Chittka L, Thomson JD (1996) The ecology of bumble bees in T-mazes. In: Elsner N, Schnitzler H (eds) Go¨ttingen Neurobiol Rep., Thieme, Stuttgart, p 130 Collett TS, Fry SN, Wehner R (1993) Sequence learning by honeybees. J Comp Physiol A 172:693–706 Couvillon PA, Biiterman ME (1988) Compound-component and conditional discrimination of colors and odors by honeybees: further tests of a continuity model. Anim Learn Behav 16:67–74 Couvillon PA, Bitterman ME (1989) Reciprocal overshadowing in the discrimination of color–odor compounds by honeybees: further tests of a continuity model. Anim Learn Behav 17:213–222 Couvillon PA, Bitterman ME (1991) How honeybees make choices. In: Goodman JL, Fischer RC (eds) The behaviour and physiology of bees. CAB International, Wallingford, pp 116–130 Dacke M, Srinivasan MV (2008) Evidence for counting in insects. Anim Cogn 11:683–689 Dehaene S (1999) The number sense: how the mind creates mathematics. Oxford University Press, Oxford Delaney PF, Reder LM, Staszewski JJ, Ritter FE (1998) The strategy specific nature of improvement: the power law applies by strategy within task. Psychol Sci 9(1):1–8 ¨ ber die Fa¨higkeit einer Schimpansin Umwege mit Do¨hl J (1968) U selbststa¨ndigen Zwischenzielen zu u¨berblicken (the ability of a female chimpanzee to overlook intermediate goals). Z Tierpsychol 25:89–103 Domjan M (2015) The principles of learning and behavior, 7th edn. Cengage Learning, Stamford Epstein R (1987) The spontaneous interconnection of four repertoires of behavior in a pigeon (Columba livida). J Comp Psychol 101:197–201 Epstein R, Kirshnit CE, Lanza RP, Rubin LC (1984) ‘Insight’ in the pigeon: antecedents and determinants of an intelligent performance. Nature 308:61–62. doi:10.1038/308061a0

123

1154 Funayama ES, Couvillon PA, Bitterman ME (1995) Compound conditioning in honeybees: blocking tests of the independence assumption. Anim Learn Behav 23:429–437 Gegear RJ, Laverty TM (1995) Effect of flower complexity on relearning flower-handling skills in bumble bees. Can J Zool 73:2052–2058 Gegear RJ, Laverty TM (1998) How many flower types can bumblebees work at the same time? Can J Zool 76:1358–1365 Giurfa M, Nu´n˜ez JA (1992) Honeybees mark with scent and reject recently visited flowers. Oecologia 89:113–117 Giurfa M, Nu´n˜ez JA (1993) Visual modulation of a scent-marking activity in the honeybee, Apis mellifera L. Naturwissenschaften 80:376–379 Grasso FW, Basil JA (2009) The evolution of flexible behavioral repertoires in cephalopod mollusks. Brain Behav Evol 74:231–245 Gross HJ, Pahl M, Si A, Zhu H, Tautz J, Zhang S (2009) Number based visual generalisation in the honeybee. PLoSONE 4(1):e4263. doi:10.1371/journal.pone.0004263 Heathcote A, Brown S, Mewhort DJK (2000) The power law repealed: the case for an exponential law of practice. Psychonom Bull Rev 7(2):185–207 Heinrich B (1979) ‘‘Majoring’’ and ‘‘minoring’’ in bumblebees: an experimental analysis. Ecology 60:245–255 James W (1890) The principles of psychology. Macmillan & Co., London Karban R, Black CA, Weinbaum SA (2000) How 17-year cicadas keep track of time. Ecol Lett 3:253–256 Kohler W (1925) The mentality of apes. 2nd ed. Harcourt, Brace & Co., New York (transl. from German by E. Winter) Lashlev KS (1938) Conditional reactions in the rat. J Psychol 6:311–324 Leppik EE (1953) The ability of insects to distinguish numbers. Amer Nat 87:229–236 Mackintosh NJ (1974) Psychology of animal learning. Academic Press, USA Martin H (1965) Leistungen des topochemischen sinnes bei der Honigbiene. Z vergl Physiol 50:254–292 Menzel R (1990) Learning, memory and ‘cognition’ in honey bees. In: Kesner RP, Olten DS (eds) Neurobiology of comparative cognition. Erlbaum Inc., Hillsdale, pp 237–292 Mirwan HB, Kevan PG (2014) Problem solving by worker bumblebees Bombus impatiens (Hymenoptera: Apoidea). Anim Cogn 17:1053–1061. doi:10.1007/s10071-014-0737-0 Mirwan HB, Kevan PG (2015) Maze learning and route memorization by worker bumblebees (Bombus impatiens Cresson (Hymenoptera: Apidae)). J Insect Behav 28:345–357

123

Anim Cogn (2015) 18:1143–1154 Mostofsky DI (1965) Stimulus generalization. Stanford University Press, Stanford Newell A, Rosenbloom PS (1981) Mechanisms of skill acquisition and the law of practice. In: Anderson JR (ed) Cognitive skills and their acquisition. Erlbaum Inc., Hillsdale, pp 1–55 North AI, Maller O, Hughes C (1958) Conditional discrimination and stimulus patterning. J Comp Physiol Psychol 51:711–715 Pahl M, Si A, Zhang S (2013) Numerical cognition in bees and other insects. Front Psychol 4:1–9. doi:10.3389/fpsyg.2013.00162 Pearce JM (2008) Animal learning and cognition: an introduction. Psychology Press, Amsterdam Ritter FE, Schooler LJ (2002) The learning curve. In: International encyclopedia of the social and behavioral sciences. Pergamon Press, Amsterdam, pp 8602–8605. http://www.iesbs.com/ Saavedra MA (1975) Pavlovian compound conditioning in the rabbit. Learn Motiv 6:314–326 Schrier AM, Thompson CR (1980) Conditional discrimination learning: a critique and amplification. J Exp Anal Behav 33:291–298 Skinner BF (1938) The behavior of organisms: AN experimental analysis. Appleton-Century-Crofts, New York Skinner BF (1953) Science and human behavior. The Macmillan Co., New York Smith MP (1951) The stimulus-trace gradient in visual discrimination learning. J Comp Physiol Psych 44:154–161 Spiegel R, McLaren IPL (2006) Associative sequence learning in humans. J Exp Psychol: Anim Behav Proc 32:156–163 Sun R (2001) Introduction to sequence learning: sequence learning. Lecture notes in computer science. Comput Sci 1828/2001 Sun R, Giles CL (2001) Sequence learning: from recognition and prediction to sequential decision making. IEEE Intelligent Systs Their Appl 16(4):2–5 Taylor AH, Elliffe D, Hunt GR, Gray RD (2010) Complex cognition and behavioural innovation in New Caledonian crows. Proc R Soc B 277:2637–2643 Terrace HS (1987) Chunking by a pigeon in a serial learning task. Nature 325:149–151 Terrace HS (1991) Chunking during serial learning by a pigeon: I. Basic evidence. J Exp Psychol: Anim Behav Proc 17:81–93 Thomas DR, Curran PJ, Russell RJ (1988) Factors affecting conditional discrimination learning by pigeons: II. Physical and temporal characteristics of stimuli. Anim Learn Behav 6:468–476 Weisman RG, Dodd PWD, Wasserman EA, Larew MB (1980) Representation and retention of two-event sequences in pigeons. J Exp Psychol: Anim Behav Proc 6:312–325