Psychopharmacology DOI 10.1007/s00213-014-3789-6

ORIGINAL INVESTIGATION

Differential effects of clozapine, metoclopramide, haloperidol and risperidone on acquisition and performance of operant responding in rats Tyson W. Baker & Matthew M. Florczynski & Richard J. Beninger

Received: 29 May 2013 / Accepted: 20 October 2014 # Springer-Verlag Berlin Heidelberg 2014

Abstract Rationale Prior research has not systematically investigated the effects of systemic antipsychotic drugs on operant response acquisition, specifically their behavioural microstructure, reinforcement blunting and relative potency in acquisition compared to performance once operant responding has stabilized. Objectives This study aims to systematically investigate the effects of systemically administered clozapine, metoclopramide, haloperidol and risperidone during free operant response acquisition and performance. Methods Following magazine training, food-restricted male Wistar rats lever pressed for food reward in 15 min daily operant conditioning sessions. Results All drugs suppressed operant response acquisition and performance. Risperidone and metoclopramide, but not clozapine or haloperidol, suppressed operant responding more potently during acquisition than performance. The dopamine D2-like receptor antagonists haloperidol and metoclopramide that affect the ventral and dorsal striatum blunted reinforcement and decreased inactive lever presses in acquisition. In contrast, the atypical antipsychotics clozapine and risperidone that affect the ventral striatum and prefrontal cortex failed to decrease inactive lever presses during acquisition, suggesting a possible decision-making deficit. Haloperidol decreased active lever pressing over performance days. The drugs did not appear to affect rats’ sensitivity to active lever press outcome, even though they suppressed active lever pressing. Electronic supplementary material The online version of this article (doi:10.1007/s00213-014-3789-6) contains supplementary material, which is available to authorized users. T. W. Baker : M. M. Florczynski : R. J. Beninger (*) Department of Psychology, Queen’s University, 62 Arch St., Kingston, ON K7L 3N6, Canada e-mail: [email protected]

Conclusions Results suggest that reinforcement impact during operant acquisition is dependent on dopamine D2 receptors while drugs affecting, among other areas, the prefrontal cortex produce a deficit in ability to suppress inactive lever press responses. Keywords Antipsychotics . Operant conditioning . Instrumental learning . Acquisition . Reinforcement (psychology) . Clozapine . Risperidone . Haloperidol . Metoclopramide

Introduction Many psychoactive drugs decrease locomotion and operant response performance for food reward, but only drugs with high dopamine (DA) D2-like (D2, D3 and D4) receptor affinity (e.g. haloperidol or metoclopramide) induce an extinctionlike decline (Beninger et al. 1987; McOmish et al. 2012; Salamone 1986; Sanger 1986; Varvel et al. 2002; Wise et al. 1978) that is dissociable from extinction from nonreinforcement upon careful examination (Beninger et al. 1987; Salamone 1986). In contrast, some drugs with low to moderate DA D2-like affinity (e.g. clozapine or risperidone) lead to tolerance with repeated administration and testing (Varvel et al. 2002). The DA D2-like receptor antagonist haloperidol or pimozide suppressed operant response acquisition (Hudzik and Palmer 1995; Tombaugh et al. 1979; Wise and Schwartz 1981), but these studies did not test if suppression was due to reinforcement blunting nor did they compare relative suppression between acquisition and performance. The present study evaluated and compared the effects of several systemically administered antipsychotic drugs on operant response

Psychopharmacology

acquisition and performance and used the results to assess possible reinforcement blunting effects of the drugs. We have three novel research questions: (1) What are the effects of systemically administered clozapine, metoclopramide, haloperidol and risperidone during operant response acquisition? (2) How do the drugs affect subsequent drug-free operant responding, i.e. is there evidence of blunted reinforcement? and (3) How do the drug effects during acquisition compare to effects during performance? Answering these questions provides information regarding pharmacologically dissociable learning and performance processes that are important for expanding scientific knowledge itself, but also important when selecting these drugs for clinical use. For example, information about these drug effects makes medication selection by clinicians better suited to a patient’s occupational or lifestyle needs for unimpaired skill acquisition and performance. We also predict operant response acquisition and performance will be differentially affected by our selected drugs based on their differential regionally preferential neural action overlapping with the differential neural regions underlying operant response acquisition and performance. We have previously reviewed the actions of antipsychotic drugs in c-fos immunohistochemical, electrophysiological, microdialysis, brain morphology and cognitive-behavioural studies focusing on their differential regional effects (Beninger et al. 2010). Metoclopramide, haloperidol and other typical antipsychotics tended to preferentially affect the dorsolateral striatum (DLS) and the ventral striatum (VS) including the nucleus accumbens. In contrast, clozapine and other atypical antipsychotic drugs (except risperidone) tended to influence the VS and prefrontal cortex (PFC). Risperidone preferentially induces c-fos in the VS compared to the DLS but does not induce c-fos in the PFC despite a high affinity for PFC 5HT-2A receptors and having other activity in the PFC (Beninger et al. 2010; Robertson et al. 1994; Schotte et al. 1993, 1996). As animals acquire instrumental responding, the PFC and VS are implicated early, then the dorsomedial striatum (DMS) as acquisition stabilizes with moderate training and finally the DLS with extensive training (Andrzejewski et al. 2004; 2005; Baldwin et al. 2000, 2002a, b; Balleine and O’Doherty 2010; Belin et al. 2009; Beninger and Ranaldi 1993; Beninger et al. 1993; Hernandez et al. 2002, 2005, 2006; Izaki et al. 1998; Jonkman and Everitt 2009; Kelley and Holahan 1997; Kelley et al. 1997; Lingawi and Balleine 2012; McKee et al. 2010; Shiflett et al. 2010; Smith-Roe and Kelley 2000; Yin et al. 2004, 2005a, b, 2008). We hypothesized that all drugs, because they affect the VS, will suppress acquisition, but clozapine and risperidone, because they have more PFC activity, will have effects dissociable from haloperidol and metoclopramide. We hypothesized that haloperidol and metoclopramide, because they affect the DLS, will disrupt performance more than clozapine and

risperidone. We expected that the DA D2-like receptor antagonists, haloperidol and metoclopramide, will blunt reinforcement during acquisition and produce an extinction-like decline in performance but that the suppression due to clozapine and risperidone will decrease over days.

Method Subjects Male Wistar rats (N=240; Charles River, St. Constant, QC) weighed 225–250 g on arrival. Seventeen rats were removed from the experiment due to technical difficulties. Clear plastic cages (40×25×22 cm high) contained two rats: an approx. 10cm-long black polyvinyl chloride tube for enrichment and 4cm-deep Beta Chip bedding (Northeastern Products Corp, Warrensberg, NY). Cages were kept in an environmentally controlled room on a reversed 12-h light/dark cycle with lights off at 7:00 a.m. Rats had water available ad libitum but were food restricted 23 h a day, allowing 1 h free access to lab chow (Purina Lab Diet 5001, St. Louis, MO) beginning 30–60 min after daily operant conditioning sessions. We followed the guidelines of the Canadian Council on Animal Care and the Animals for Research Act, and procedures were approved by the Queen’s Univ. Animal Care Committee. Apparatus We used four operant conditioning chambers (26.5×22× 20 cm high). The back and side walls were stainless steel, the front wall and ceiling were clear Plexiglas, and the floor was 3 mm diameter stainless steel rods spaced 11 mm apart. Opposing side walls had two retractable levers 3.5 cm wide; lever activity (active vs. inactive) was counterbalanced across operant conditioning chambers. The back wall contained two cue lights and a feeder magazine equipped with a beam to detect nose pokes. The feeder dispensed 45 mg dustless food pellets (Bioserv, Frenchtown, NJ). The operant conditioning chambers were housed in painted, wooden enclosures each outfitted with a ventilation fan and 15×15 cm observation window. Drugs Clozapine, haloperidol and risperidone (Sigma, Oakville, ON) were dissolved in 0.3 % tartaric acid and 0.9 % saline solution. Haloperidol was heated until dissolved. Metoclopramide (Sigma, Oakville, ON) was dissolved in distilled water. We injected 1 ml/kg intraperitoneally (i.p.) 30 min (0.0–3.0 mg/kg clozapine) or 60 min (0.000–0.200 mg/kg haloperidol, 0– 10 mg/kg metoclopramide, 0.00–0.80 mg/kg risperidone) prior to operant conditioning sessions, similar to previous

Psychopharmacology

behavioural studies (Banasikowski et al. 2010; Beninger et al. 1987; Dunn and Killcross 2006; Goetghebeur and Dias 2009; Varvel et al. 2002). Procedure Our operant response acquisition protocol is based on protocols used previously (e.g. Andrzejewski et al. 2004, 2005; Kelley et al. 1997; Kelley and Holahan 1997). Magazine training During three daily 15 min operant conditioning sessions with the levers retracted, pellets were presented according to a random time (RT) 30 s schedule. Cue lights illuminated for 500 ms prior to pellet delivery throughout the experiment. Doses in acquisition were assigned balancing magazine training nose poke rate, operant conditioning chamber and time of testing. Acquisition Rats received 10 daily 15 min operant conditioning sessions. With the exception of non-injected control rats, rats received injections on days 1–4 and 10. Operant conditioning sessions began when both levers were inserted and a pellet was dispensed. The active lever was programmed on continuous reinforcement for the first 20 pellets of each operant conditioning session, then random ratio 2 for the remainder of the session. Both levers retracted at the completion of the session. We placed crushed pellets on both levers only during days 1–2 to expedite operant response acquisition. Response shaping was never used. We measured nose pokes, active lever presses and inactive lever presses as well as what occurred prior to these behaviours (non-reinforced active lever press, reinforced active lever press, inactive lever press or nose poke) to understand the effects of pharmacological manipulations on the microstructural pattern of behaviour (see supplemental methods). Extended training Rats received five daily 30 min operant conditioning sessions. Performance Rats received injections prior to each of five daily 15 min operant conditioning sessions. Doses were assigned balancing lever press rate during extended training, along with time of day, operant chamber (and active lever) and acquisition doses.

Data analysis General analysis We use polynomial contrasts for analyses over days to detect and characterize overall patterns in the data (linear, quadratic, cubic) that might not be detected by other analyses. These patterns can occur simultaneously, are orthogonal and do not require sphericity of variance (Glass and Hopkins 1996). Unless specified otherwise, we have used dose as a covariate using 1 degree of freedom because dose is a scale variable, differing in degree, not differing in type. Acquisition-performance potency comparison To account for differences between vehicle response rate between acquisition and performance, we converted each rats’ response rate into percent vehicle, including vehicle rats. Vehicle groups had a defined mean of 100 %, so they were only included in the dose × group interaction (Dawson and Richter 2006). Analysis comparing 4 days of acquisition to the first 4 days of performance yielded similar interpretations as analysis of acquisition day 4 and performance day 4 (data not shown) so only acquisition day 4 and performance day 4 (experimental day 19) are presented for simplicity. Reinforcement impact analysis Drug-free responding allows measurement of learned reinforcement impact free of concurrent drug effects, such as motor or motivational effects (Yin et al. 2005a, 2008). We statistically controlled for reinforcement history by calculating the mean cumulative pellets earned by the end of day 4 for each dose group. Next, we matched the mean reinforcement history of each dose group using the smallest absolute difference in cumulative pellets prior to day 5 from individual vehicle rats. We used reinforcement history to predict subsequent drugfree (day 5 for drug groups, the next day for reinforcementmatched vehicle rats) lever press rate and if drug exposure during reinforcement history affected this slope as calculated by Dawson and Richter (2006). The linear contrast dose– k hpffiffiffiffiffiffiffiffi 2 i response used SSerror ¼ ∑ n−1  SD on N−K−1 degree of freedom. We interpret a lesser slope between reinforcement history and subsequent drug-free lever press rate to indicate blunted reinforcement impact. This directly tests if dose during reinforcement history altered the role of reinforcement history on subsequent drug-free responding independent of group differences in lever press rate while controlling reinforcement history. An alternative explanation for reinforcement blunting must explain the interaction of reinforcement

Psychopharmacology

history and previous drug (slope difference), not previous drug effects alone (mean difference) (e.g. Yin et al. 2005a).

Results For drug-free operant response acquisition microstructure, see supplementary results, Table S1 and Fig. S1. For the effect of the following drugs on operant response acquisition microstructure, see supplementary results, Table S2 and Fig. S2-S5.

Clozapine Acquisition Lever pressing increased over the first 4 days (linear: F(1, 63) = 385.13, p < .001; quadratic: F(1, 63)=25.03, p<.001) and clozapine (Fig. 1a, S6) suppressed lever pressing (F(1, 63)=9.74, p=.003). This suppression appeared to increase over days, but the interaction only approached significance (F(1, 63)=3.32, p=.073). Clozapine did not have a significant residual effect on day 5 or the entire day 5–9 period (F values <1.3, p values >.270), so reinforcement blunting analyses were not conducted.

Day 10 vs. drug-free day 9 There was a significant day × dose interaction (F(1, 63)= 141.37, p<.001) indicating that clozapine doses of 0.5 mg/kg and above suppressed lever pressing on day 10 compared to day 9 (F values >7.65, p values <.020; Fig. 1a), but 0.3 mg/kg did not (F<0.50, p>.500). The vehicle rats showed a nearsignificant increase (F(1,13)=4.38, p=.057).

Days 11–15 extended training There was no residual main effect of any drug over days (data not shown; F values <1.00, p values >.440). Dose reassignments of the drugs were performed in a manner that balanced responding on these 5 days (data not shown; F values <1.00, p values >.400). Performance Additional control rats (n=48) did not receive injections during acquisition (see microstructure in supplementary methods, results, Table S1, and Fig. S1) but received injections during performance. These rats produced performance results similar to rats that received clozapine during acquisition, so the groups were pooled. In fact, no acquisition dose for any drug tested had a significant effect or interaction when added to the performance analyses (data not shown; F values <1.00, p values >.440), indicating that drugs in acquisition did not significantly affect performance itself or performance response to drugs. Because there was not even a suggestive effect of any acquisition drug dose on performance and to conserve animals, we did not use non-injected rats for testing the effects of the other drugs in performance. Clozapine suppressed lever pressing (main effect: F(1, 111)=43.34, p<.001; Fig. 1b) and rats increased operant responding over days (linear main effect: F(1, 111)=5.393, p=.022) as rats developed tolerance, demonstrating less suppression in the later days (clozapine × linear day interaction: F(1, 111)=26.50, p<.001). Acquisition-performance potency comparison Clozapine was similarly potent in acquisition and performance (phase: F(1, 137)=1.37, p=.245; phase × dose: F(1, 174)=0.44, p=.507; Fig. 2a). Summary Clozapine suppressed operant responding in acquisition and performance with similar potency. Rats developed tolerance to clozapine during performance. Clozapine did not appear to suppress instrumental learning itself, but rather seemed to have general suppressive effects that gradually diminished, showing tolerance. Metoclopramide Acquisition

Fig. 1 Mean (±SEM) daily operant responses after clozapine (0.0– 3.0 mg/kg i.p.) administration during operant response acquisition (a) and performance after extended training (b)

Operant responding increased over the first 4 days (linear: F(1, 22) = 179.48, p < .001; quadratic: F(1, 22) = 20.17, p < .001), and metoclopramide (Fig. 3a, S7) suppressed

Psychopharmacology

Day 10 vs. drug-free day 9 Previous metoclopramide still reduced operant responding on day 9 (F(1, 22)=6.88, p=.016). Metoclopramide decreased operant responding from day 9 to 10 (day × dose: F(1, 22)= 122.88, p<.001). Operant responding decreased from day 9 to 10 in the 10 mg/kg (F(1, 5)=319.67, p<.001) and 5 mg/kg (F(1, 5)=9.93, p=.025), but not 1 mg/kg or vehicle groups (F values <0.30, p values >.610; Fig. 3a).

Performance Metoclopramide suppressed operant responding (dose: F(1, 22)=178.74, p<.001; Fig. 3b) and rates did not change significantly over days (F values <1.10, p values >.310).

Acquisition-performance potency comparison Fig. 2 Mean (±SEM) % vehicle operant response rate during acquisition and performance following clozapine (a), metoclopramide (b), haloperidol (c), and risperidone (d) administration. n values are the same as the respective Figs. 1, 3, 4, and 6

operant responding (dose: F(1, 22)=65.25, p<.001, dose × linear day: F(1, 22)=73.85, p<.001). The 0, 1 and 5 mg/kg dose groups increased over days (F values >4.00, p values <.003) but not the 10 mg/kg group (F<2.40, p>.180). Prior exposure to metoclopramide suppressed operant responding on day 5 (F(1,22)=43.54, p<.001; Fig. 3a). Ten milligrams per kilogram suppressed operant responding too much to match reinforcement history without using zero reinforcement history, so it was not analysed. Metoclopramide significantly blunted reinforcement (linear: F(1, 16)=33.95, p<.001; Fig. 5a)

There was a floor effect using 10 mg/kg so we removed this dose from the analyses (Fig. 2b). Metoclopramide more potently decreased operant responding during acquisition (main effect of phase: F(1,21)=10.94, p=.003).

Summary Metoclopramide suppressed operant responding in acquisition more potently than performance. Metoclopramide suppressed acquisition operant responding more potently in later days compared to early days, suggesting sensitization over days. Metoclopramide also decreased the slope of the reinforcement history-drug-free responding regression line, suggesting this was at least in part due to metoclopramide suppressing lever press learning by reinforcement blunting.

Haloperidol Acquisition

Fig. 3 Mean (±SEM) daily operant responses after metoclopramide (0– 10 mg/kg i.p.) administration during operant response acquisition (a) and performance after extended training (b)

Operant responding increased over the first 4 days (linear: F(1, 39)=139.39, p<.001), which haloperidol (Fig. 4a, S8) suppressed (F(1, 39)=66.94, p<.001), especially in later days (dose × linear day: F(1, 39)=56.42, p<.001). Rats acquired operant responding over days in doses 0.075 mg/kg and lower (F values >7.00, p values <.045), but not in doses of 0.100 mg/kg and higher (F values <3.50, p values >.130). Previous haloperidol exposure decreased operant responding on day 5 (F(1, 39)=32.40, p<.001; Fig. 4a) and significantly blunted reinforcement (linear: F(1, 39)=35.51, p<.001; Fig. 5b).

Psychopharmacology

(linear: F(1, 10)=5.55, p=.040) and 0.025 mg/kg dose (linear: F(1, 5)=6.21, p=.055) and decreases in the 0.050 (linear: F(1, 4)=50.76, p=.002), 0.100 (linear: F(1, 2)=10.65, p=.082) and 0.200 mg/kg doses (linear: F(1, 4)=4.79, p=.094).

Acquisition-performance potency comparison Haloperidol was similarly potent during acquisition and performance (F values <2.30, p values >.130; Fig. 2c)

Summary Fig. 4 Mean (±SEM) daily operant responses after haloperidol (0.000– 0.200 mg/kg i.p.) administration during operant response acquisition (a) and performance after extended training (b)

Haloperidol showed similar potency during acquisition and performance. During acquisition, haloperidol blocked lever press learning in a manner that suppressed the reinforcing properties of the pellets. Some doses of haloperidol produced an extinction-like decline in performance.

Day 10 vs. drug-free day 9 Previous haloperidol dose had no significant effect on day 9 (F < 1.40, p > .250). Haloperidol suppressed operant responding from day 9 to 10 (day × dose: F(1, 39)=84.77, p<.001) which was significant at 0.200 mg/kg (F(1, 3)= 35.15, p=.010) and 0.150 mg/kg (F(1, 4)=31.25, p=.005), marginally significant at 0.075 mg/kg (F(1, 5)=4.81, p=.080), but not by other doses (Fig. 4a).

Risperidone Acquisition

0.5

0.5

0

0

0

-0.5

-0.5

-0.5

-1

-1

-1

-1.5

-1.5

-1.5

0

1

2

3

4

mg/kg Meto

5

mg/kg Hal

mg/kg Risp

0.8

0.5

0.6

1

0.20

1

1

0.15

1.5

1.5

0.10

c

0.05

b 1.5

0.00

a Dose Slope - Vehicle Slope

Fig. 5 Data points are the slope difference between vehicle groups and respective dose groups of metoclopramide (a), haloperidol (b) and risperidone (c), and are based on the same data as Figs. 3, 4 and 6. Error bars are for visual aid only and indicate the pooled SEMs used for the regression calculation

0.4

Haloperidol suppressed operant responding (dose: F(1, 39)= 148.44, p<.001; Fig. 4b) and there was no main effect of day (F values <1.35, p values >.250). A significant dose × day interaction (linear: F(6, 34)=7.40, p<.001; quadratic: F(6, 34)=2.97, p=.019) reflected significant or suggestive increases in operant responding over days in the vehicle

0.2

Performance

0.0

Rats acquired operant responding over the first 4 days (linear: F(1, 43)=175.33, p<.001; quadratic: F(1, 43)= 12.78, p=.001), which risperidone suppressed (F(1, 43)= 60.63, p<.001; Fig. 6a, S9), especially in later days (dose × linear day: F(1, 43)=28.85, p<.001). Previous risperidone exposure decreased operant responding on day 5 (F(1, 43)=26.03, p<.001; Fig. 6a) but did not significantly blunt reinforcement (F<0.30, p>.600; Fig. 5c). The apparent increase at 0.3 and 0.4 mg/kg is due to an atypically negative slope in the risperidone vehicle control data at this reinforcement history.

Psychopharmacology

Acquisition-performance potency comparison Risperidone suppressed operant responding more potently during acquisition (dose × phase: F(1, 86)=4.65, p=.034; phase: F(1, 63)=7.37, p=.009; Fig. 2d). Summary

Fig. 6 Mean (±SEM) daily operant responses after risperidone (0.00– 0.80 mg/kg i.p.) administration during operant response acquisition (a) and performance after extended training (b)

Risperidone was more potent in acquisition than performance. This suppression appeared stronger is later acquisition days, which may have been due to the increased operant responding over days in vehicle rats. There was no evidence of blunted reinforcement learning. Rats developed tolerance to risperidone over performance days. Operant responding results from the four drugs are summarized in Table 1.

Day 10 vs. drug-free day 9

Discussion

Previous risperidone dose did not suppress operant responding on day 9 (F<2.70, p>.110). All doses significantly suppressed operant responding on day 10 (day × dose: F(1, 39)=84.77, p<.001; Fig. 6a; all day effects within dose group F values >17.30, p values <.026) except 0.60 mg/kg which was nearly significant (F(1, 5)=5.578, p=.065); vehicle did not change significantly (F<.10, p>.780).

Operant response performance with systemic drug treatment

Performance Risperidone suppressed operant responding (dose: F(1, 43)= 112.60, p<.001; Fig. 6b) and rates increased over days (linear day: F(1, 43)=8.51, p=.006; quadratic: F(1, 43)= 12.95, p=.001). Tolerance developed to risperidone (dose × linear day: F(1, 43)=6.05, p=.018; quadratic: F(1, 43)= 6.55, p=.014). Nearly all groups increased significantly over days (F values >8.90, p values <.035) with the exception of near-significant increases in the 0.30 mg/kg (linear: F(1, 5)= 5.79, p=.061) and vehicle (linear: F(1, 11)=4.74, p=.052) groups. The 0.80 mg/kg group showed a significant increase and decrease, peaking at day 3 (quadratic: F(1, 4)= 21.23, p=.010).

Table 1 Summary of drug effects in operant acquisition and performance Clozapine Metoclopramide Haloperidol Risperidone

Risperidone and clozapine each suppressed operant response performance early, with decreased suppression over days suggesting tolerance, consistent with previous research (Varvel et al. 2002). Haloperidol suppressed operant response performance with increased suppression over days, consistent with previous research (Salamone 1986; Sanger 1986; Varvel et al. 2002). Metoclopramide suppressed operant response performance, but not incrementally over days as previously reported (Beninger et al. 1987), possibly as a result of a floor effect. The D2-like receptor antagonist pimozide produces a gradual decrease over days (Wise et al. 1978), suggesting that blocking D2-like receptors produces this effect. This effect is extinctionlike and can decrease subsequent drug-free operant responding similar to extinction (Johnston et al. 2001), but D2 antagonism suppresses operant response performance in a manner dissociable from extinction due to non-reinforcement (Beninger et al. 1987; Salamone 1986). The apparent blunting of reinforcement in pre-trained rats by metoclopramide and haloperidol versus risperidone and clozapine is consistent with an action of the typicals but not the atypicals on the DLS (e.g. Beninger et al. 2010; Belin et al. 2009).

Potency in acquisition vs. performance

Acquisition learning

Performance pattern

Equipotent Acquisition greater Equipotent Acquisition greater

No effect Blocked/suppressed Blocked/suppressed No effect

Tolerance None meaningful Sensitization at 0.050 mg/kg Tolerance (except highest dose)

Psychopharmacology

Operant response acquisition with systemic drug treatment Our general finding of drugs suppressing acquisition is consistent with previous literature using DA D2-like receptor antagonists and anticonvulsants (Hudzik and Palmer 1995; Tombaugh et al. 1979; Wise and Schwartz 1981). To our knowledge, we are the first to demonstrate that only drugs with strong DA D2-like receptor antagonism, i.e. haloperidol and metoclopramide, given during acquisition produced lasting effects that were in part attributable to blunted reinforcement and were not due to differential number of previous reinforcers or residual drug effects. Decreased lever press rate during drug acquisition days is not evidence of reinforcement blunting (Yin et al. 2005a, 2008). Group differences in subsequent drug-free lever press rate have been used in an attempt to isolate instrumental learning despite some experimental groups receiving barely half the reinforcement history of other groups (e.g. Yin et al. 2005a, Exp 3). We improve upon this approach in two ways: one, we statistically tightly control reinforcement history. Two, we test if experimental manipulations change the effect of reinforcement history. It is the interaction of drug and reinforcement history that is evidence of reinforcement blunting, not just group differences in mean lever press rate. Suggested alternative explanations such as effects on response initiation, motivation, malaise, motoric impairment, changes in receptor sensitivity or effort-related impairments may explain group differences in mean lever press rate, but do not explain the interaction with reinforcement history (see Salamone and Correa 2012 for a recent review of potential explanations). Future research may utilize post-operant conditioning session drug administration to test if these effects are attributable to drug activity within the operant conditioning session or consolidation processes. It is unclear why metoclopramide and risperidone, but not haloperidol or clozapine demonstrated stronger potency in acquisition compared to performance. To our knowledge, our study is the first to examine the microstructure of behaviour following systemically administered drugs during free operant acquisition, allowing measurement of specific processes separate from general suppression. All drugs suppressed active lever pressing, decreasing the number of reinforcements, decreasing nose pokes to retrieve reinforcements. Only clozapine made nose pokes less selective by increasing nose pokes following nose pokes: checking the presumably empty food magazine. Nose pokes into the magazine were otherwise preferential following reinforced active lever presses compared to non-reinforced active lever presses, indicating no drug affected action-outcome contingencies. The atypical antipsychotic drugs with stronger PFC action, clozapine and risperidone, delayed the normal decrease in inactive lever presses early in acquisition, suggesting a possible deficit in very basic (reinforced vs. non-reinforced) decision-making similar to the deficit in complex decision-

making in the Iowa Gambling Task found in schizophrenia patients treated with clozapine (Wasserman et al. 2012) and a rodent version of the Iowa Gambling Task (unpublished data). In contrast, the D2 receptor-preferring antagonists, haloperidol and metoclopramide, suppressed inactive lever presses, indicating no deficit in basic decision-making (McCormick et al. 2010; Robertson et al. 1994; Schotte et al. 1993, 1996).

Conclusion Dopamine D2-like receptor antagonists blunt the reinforcing impact of food reward during operant response acquisition. In contrast, atypical antipsychotics with relatively preferential action in the PFC do not blunt reinforcement impact but delay the decrease in inactive lever pressing selectivity during operant acquisition, indicating a possible decision-making deficit. Acknowledgments We would like to thank Drs. Hans Dringenberg, Janet Menard and M. Cella Olmstead for their valuable advice on this project and manuscript. This study was funded by grant no. 7861-2010 from the Natural Sciences and Engineering Research Council of Canada to RJB.

References Andrzejewski ME, Sadeghian K, Kelley AE (2004) Central amygdalar and dorsal striatal NMDA receptor involvement in instrumental learning and spontaneous behavior. Behav Neurosci 118:715–729 Andrzejewski ME, Spencer RC, Kelley AE (2005) Instrumental learning, but not performance, requires dopamine D1-receptor activation in the amygdala. Neuroscience 135:335–345 Baldwin AE, Holahan MR, Sadeghian K, Kelley AE (2000) N-Methyl-Daspartate receptor-dependent plasticity within a distributed corticostriatal network mediates appetitive instrumental learning. Behav Neurosci 114:84–98 Baldwin AE, Sadeghian K, Holahan MR, Kelley AE (2002a) Appetitive instrumental learning is impaired by inhibition of cAMP-dependent protein kinase within the nucleus accumbens. Neurobiol Lern Mem 77:44–62 Baldwin AE, Sadeghian K, Kelley AE (2002b) Appetitive instrumental learning requires coincident activation of NMDA and dopamine D1 receptors within the medial prefrontal cortex. Neuroscience 22: 1063–1071 Balleine BW, O’Doherty JP (2010) Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology 35:48–69 Banasikowski TJ et al (2010) Double dissociation of the effects of haloperidol and the dopamine D3 receptor antagonist ABT-127 on acquisition vs. expression of cocaine-conditioned activity in rats. J Pharmacol Exp Ther 335:506–515 Belin D et al (2009) Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res 199:89–102 Beninger RJ, Ranaldi R (1993) Microinjections of flupenthixol into the caudate-putamen but not the nucleus accumbens, amygdala or

Psychopharmacology frontal cortex of rats produce intra-session declines in foodrewarded operant responding. Behav Brain Res 55:203–212 Beninger RJ et al (1987) Effects of extinction, pimozide, SCH 23390, and metoclopramide on food-rewarded operant responding of rats. Psychopharmacology 92:343–349 Beninger RJ, D’Amico C, Ranaldi R (1993) Microinjections of flupenthixol into the caudate putamen of rats produce intrasession declines in food-rewarded operant responding. Pharmacol Biochem Behav 45:343–350 Beninger RJ, Baker TW, Florczynski MM, Banasikowski TJ (2010) Regional differences in the action of antipsychotic drugs: implications for cognitive effects in schizophrenic patients. Neurotox Res 18:229–243 Dawson JF, Richter AW (2006) Probing three-way interactions in moderated multiple regression: development and application of a slope difference test. J Appl Psychol 91:917–926 Dunn MJ, Killcross S (2006) Clozapine but not haloperidol treatment reverses sub-chronic phencyclidine-induced disruption of conditional discrimination performance. Behav Brain Res 175:271–277 Glass GV, Hopkins KD (1996) Statistical methods in education and psychology, 3rd edn. Allyn and Bacon, Boston Goetghebeur P, Dias R (2009) Comparison of haloperidol, risperidone, sertindole, and modafinil to reverse an attentional set-shifting impairment following subchronic PCP administration in the rat—a back translational study. Psychopharmacology 202:287–293 Hernandez PJ, Sadeghian K, Kelley AE (2002) Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens. Nat Neurosci 5:1327–1331 Hernandez PJ et al (2005) AMPA/kainate, NMDA, and dopamine D1 receptor function in the nucleus accumbens core: a context-limited role in the encoding and consolidation of instrumental memory. Learn Mem 12:285–295 Hernandez PJ, Schiltz CA, Kelley AE (2006) Dynamic shifts in corticostriatal expression patterns of the immediate early genes Homer 1a and Zif268 during early and late phases of instrumental training. Learn Mem 13:599–608 Hudzik TJ, Palmer GC (1995) Effects of anticonvulsants in a novel operant learning paradigm in rats: comparison of remacemide hydrocholoride and FPL 15896AR to other anticonvulsant agents. Epilepsy Res 21:183–193 Izaki Y, Hori K, Nomura M (1998) Dopamine and acetylcholine elevation on lever-press acquisition in rat prefrontal cortex. Neurosci Lett 258: 33–36 Johnston LD, Beninger RJ, Olmstead MC (2001) Pimozide, like extinction, devalues stimuli associated with sucrose taking. Pharmacol Biochem Behav 68:583–590 Jonkman S, Everitt BJ (2009) Post-learning infusion of anisomycin into the anterior cingulate cortex impairs instrumental acquisition through an effect on reinforcer valuation. Learn Mem 16:706–713 Kelley AE, Holahan MR (1997) Enhanced reward-related responding following cholera toxin infusion into the nucleus accumbens. Synapse 26:46–54 Kelley AE, Smith-Roe SL, Holahan MR (1997) Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. Proc Natl Acad Sci U S A 94: 12174–12179 Lingawi NW, Balleine BW (2012) Amygdala central nucleus interacts with dorsolateral striatum to regulate the acquisition of habits. J Neurosci 32:1073–1081 McCormick PN et al (2010) The antipsychotics olanzapine, risperidone, clozapine, and haloperidol are D2-selective ex vivo but not in vitro. Neuropsychopharmacology 35:1826–1835

McKee BL, Kelley AE, Moser HR, Andrzejewski ME (2010) Operant learning requires NMDA-receptor activation in the anterior cingulate cortex and dorsomedial striatum, but not in the orbitofrontal cortex. Behav Neurosci 124:500–509 McOmish CE, Lira A, Hanks JB, Gingrich JA (2012) Clozapine-induced locomotor suppression is mediated by 5-HT2A receptors in the forebrain. Neuropsychopharmacology 37:2747–2755 Robertson GS, Matsumura H, Fibiger HC (1994) Regional differences in the action of antipsychotic drugs: implications for cognitive effects in schizophrenic patients. Pharmacol Exp Ther 271:1058–1066 Salamone JD (1986) Different effects of haloperidol and extinction on instrumental behaviours. Psychopharmacology 88:18–23 Salamone JD, Correa M (2012) The mysterious motivational functions of mesolimbic dopamine. Neuron 76:470–485 Sanger DJ (1986) Response decrement patterns after neuroleptic and nonneuroleptic drugs. Psychopharmacology 89:98–104 Schotte A, Janssen PF, Megens AA, Leysen JE (1993) Occupancy of central neurotransmitter receptors by risperidone, clozapine and haloperidol, measured ex vivo by quantitative autoradiography. Brain Res 631:191–202 Schotte A et al (1996) Risperidone compared with new and reference antipsychotic drugs: in vitro and in vivo receptor binding. Psychopharmacology 124:57–73 Shiflett MW, Brown RA, Balleine (2010) Acquisition and performance of goal-directed instrumental actions depends on ERK signaling in distinct regions of dorsal striatum in rats. J Neurosci 30: 2951–2959 Smith-Roe SL, Kelley AE (2000) Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. Neuroscience 20: 7737–7742 Tombaugh TN, Tombaugh J, Anisman H (1979) Effects of dopamine receptor blockade on alimentary behaviors: home cage food consumption, magazine training, operant acquisition, and performance. Psychopharmacology 66:219–225 Varvel SA et al (2002) Effects of antipsychotic drugs on operant responding after acuse and repeated administration. Psychopharmacology 160: 182–191 Wasserman JI, Barry RJ, Bradford LB, Delva NJ, Beninger RJ (2012) Probabilistic classification and gambling in patients with schizophrenia receiving medication: comparison of risperidone, olanzapine, clozapine and typical antipsychotics. Psychopharmacology 222: 173–183 Wise RA, Schwartz HV (1981) Pimozide attenuates acquisition of leverpressing for food in rats. Pharmacol Biochem Behav 15:655–656 Wise RA, Spinder J, DeWit H, Gerber GJ (1978) Response decrement patterns after neuroleptic and non-neuroleptic drugs. Science 201: 262–264 Yin HH, Knowlton BJ, Balleine BW (2004) Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J NeuroSci 19:181–189 Yin HH, Knowlton BJ, Balleine BW (2005a) Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J NeuroSci 22: 505–512 Yin HH, Ostlund SB, Knowlton BJ, Balleine BW (2005b) The role of the dorsomedial striatum in instrumental conditioning. Eur J NeuroSci 22:513–523 Yin HH, Ostlund SB, Balleine BW (2008) Reward-guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico-basal ganglia networks. Eur J NeuroSci 28: 1437–1448