Classical Conditioning and Operant Conditioning

views updated

Classical Conditioning and Operant Conditioning

The simple nervous system and the relatively large identifiable neurons of the marine mollusk Aplysia provide a useful model system in which to examine the cellular mechanisms of two forms of associative learning: classical conditioning and operant (instrumental) conditioning.

Classical Conditioning

Classical conditioning occurs when an animal learns to associate a typically neutral stimulus with a later salient event. If the neutral stimulus precedes the salient event with a fixed latency, then the animal learns that the stimulus can serve as a predictor.

Behavioral Studies

A tactile or electrical stimulus delivered to the siphon results in a reflex withdrawal of the gill and siphon, which presumably serves the defensive role of protecting these sensitive structures from potentially harmful stimuli. The reflex exhibits several simple forms of learning, including habituation, sensitization, and classical conditioning. In studies of aversive classical conditioning, the conditioned stimulus (CS) is a brief, weak tactile stimulus to the siphon, which by itself produces a small siphon withdrawal. The unconditioned stimulus (US) is a short-duration, strong (noxious) electric shock to the tail, which by itself produces a large withdrawal of the siphon (the unconditioned response, UR). After repeated pairings, the ability of the CS to produce siphon withdrawal (the conditioned response, CR) is enhanced beyond that produced by presentations of the US alone (sensitization control) or of explicitly unpaired or random presentations of the CS and the US (Carew, Walters, and Kandel, 1981). The conditioning persists for as long as four days. Thomas J. Carew and colleagues (Carew, Hawkins, and Kandel, 1983) also found that this reflex exhibits differential classical conditioning. Differential classical conditioning can be produced by delivering one CS to the siphon and another to the mantle region. One CS is paired with the US (the CS+) and the other is explicitly unpaired (the CS-). As in the previous studies, the US is an electric shock delivered to the tail. After conditioning, the CS+ will produce more withdrawal than the CS-.

Other behaviors of Aplysia can also be classically conditioned. For example, feeding behavior can be classically conditioned with an appetitive protocol (Lechner, Baxter, and Byrne, 2000).

Neural Mechanisms of Aversive Classical Conditioning in Aplysia

A cellular mechanism called activity-dependent neuromodulation contributes to associative learning in Aplysia (Hawkins, Abrams, Carew, and Kandel, 1983; Walters and Byrne, 1983; Antonov, Antonova, Kandel, and Hawkins, 2001). A general cellular scheme of activity-dependent neuromodulation is illustrated in Figure 1. Two sensory neurons (SN1 and SN2), which constitute the pathways for the conditioned stimuli (CS+ and CS-), make weak subthreshold connections to a motor neuron. Delivering a reinforcing or unconditioned stimulus (US) alone has two effects. First, the US activates the motor neuron and produces the unconditioned response (UR). Second, the US activates a diffuse modulatory system that nonspecifically enhances transmitter release from all the sensory neurons. This nonspecific enhancement contributes to sensitization. Temporal specificity, characteristic of associative learning, occurs when there is pairing of the CS (spike activity in SN1) with the US, causing a selective amplification of the modulatory effects in SN1. Unpaired activity does not amplify the effects of the US in SN2. The amplification of the modulatory effects in SN1 leads to an enhancement of the ability of SN1 to activate the motor neuron and produce the CR.

As discussed in the entry "Molecular Basis of Long-Term Sensitization," experimental analyses of sensitization of defensive reflexes in Aplysia have shown that the neuromodulator released by the reinforcing stimulus, which is believed to be serotonin, activates the enzyme adenylyl cyclase in the sensory neuron and thereby increases the synthesis of the second messenger cAMP, which activates the cAMP-dependent protein kinase; the subsequent protein phosphorylation leads to a modulation in several properties of the sensory neurons. These changes in the sensory neuron include modulation of membrane conductances and other processes, which facilitate synaptic transmission. This facilitation results in the increased activation of the motor neuron and, thus, sensitization of the reflex. The pairing specificity of the associative conditioning is due, at least in part, to an increase in the level of cAMP beyond that produced by serotonin alone (Abrams and Kandel, 1988; Ocorr, Walters, and Byrne, 1985). The influx of Ca⁺² associated with the CS (spike activity) amplifies the US-mediated modulatory effect by interacting with a Ca⁺²-sensitive component of the adenylyl cyclase (Abrams and Kandel, 1988; Schwartz et al., 1983). A critical role for Ca⁺²-stimulated cyclase is also suggested by studies of Drosophila showing that the adenylyl cyclase of a mutant deficient in associative learning exhibits a loss of Ca⁺²/calmodulin sensitivity.

There are contributions to the plasticity of the synapse occurring in the motor neuron, as well (Murphy and Glanzman, 1997; Bao, Kandel, and Hawkins, 1998). The postsynaptic membrane of the motor neuron contains NMDA receptors. If these receptors are blocked, then the associative modification of the synapse is disrupted. NMDA receptors require concurrent delivery of glutamate and depolarization in order to allow the entry of calcium. Activity in the sensory neuron (CS) provides the glutamate and the US depolarizes the cell. The subsequent increase in intra-cellular Ca⁺² putatively releases a retrograde signal from the postsynaptic cell to the presynaptic terminal. This retrograde signal would then act to further enhance the cAMP cascade in the sensory neuron.

An important conclusion is that this mechanism for associative learning is an elaboration of a process already in place that mediates sensitization, a simpler form of learning. This finding raises the interesting possibility that even more complex forms of learning may use simpler forms as building blocks, an idea that has been suggested by some psychologists for many years but one that until the early years of the twenty-first century has not been testable at the cellular level.

Operant Conditioning

Operant conditioning is a process by which an animal learns the consequences of its own behavior. In an operant-conditioning paradigm, the delivery of a reinforcing stimulus is contingent upon the expression of a designated behavior. The probability of expression of this behavior will then be altered. In other words, the animal learns to associate the behavior with the contingent reinforcement and modifies its behavior accordingly.

Behavioral Studies

Feeding behavior in Aplysia has been used to gain insights into the modification of a behavioral response by reinforcement. Aplysia feed by protracting a toothed structure called the radula into contact with seaweed. The radula grasps seaweed by closing and retracting. This sequence results in the ingestion of the seaweed. Inedible objects can be rejected if the radula protracts while closed (grasping the object) and then opens as it retracts, releasing the object. Thus, the timing of radula closure determines which behavior will be elicited.

Feeding behavior in Aplysia can be modified by pairing feeding with an aversive stimulus. In the presence of food wrapped in a tough plastic net, Aplysia bite and attempt to swallow the food. However, netted food cannot be swallowed, and it is rejected. The inability to consume food appears to be an aversive stimulus that modifies feeding behavior, since trained animals do not attempt to bite netted food (Susswein, Schwarz, and Feldman, 1986).

Feeding behavior can also be operantly conditioned with an appetitive stimulus (Brembs et al., 2002). Animals receiving positive reinforcement that is contingent on biting will learn to bite more than animals receiving either reinforcement that is not contingent on their behavior or no reinforcement at all.

Neural Mechanisms of Appetitive Operant Conditioning in Aplysia

The feeding system of Aplysia has many advantages. For example, much of the cellular circuitry controlling feeding behavior has been identified. Thus, it is possible for researchers to study neurons with known behavioral significance. One of these neurons, denoted as B51, is implicated in the expression of ingestive behavior. B51 is active predominately during the retraction phase. Furthermore, when B51 is recruited into a pattern, it recruits radula closure motor neurons (see Figure 2).

An in vitro analogue of operant conditioning has been developed using only the isolated buccal ganglia, which is responsible for generating the motor programs involved in feeding (Nargeot, Baxter, and Byrne, 1999a). The ganglion expresses motor patterns that are analogous to feeding behavior. These motor patterns can either be ingestionlike or rejectionlike and the type that will be expressed is not predictable. In the analogue of operant conditioning, motor patterns corresponding to ingestion are selectively reinforced by contingently shocking the esophageal nerve. The esophageal nerve contains dopaminergic processes and is believed to be part of the pathway mediating food reward. The conditioning results in an increase in the likelihood of ingestionlike patterns being produced. The contingent reinforcement also results in the modulation of the membrane properties of neuron B51. The input resistance increases and a smaller stimulus is required to elicit electrical activity. Thus, the cell is more likely to be active in the future. This change in the likelihood of B51 activation contributes to the conditioned increase in ingestionlike patterns. Furthermore, these results can be replicated when induced electrical activity in B51 is substituted as the analogue of the behavior, instead of an ingestionlike motor pattern (Nargeot, Baxter, and Byrne, 1999b).

The analogue can be further reduced by removing neuron B51 from the ganglia and placing it in culture (Lorenzetti, Baxter, and Byrne, 2000; Brembs et al., 2002). This single, isolated neuron can be conditioned by contingently reinforcing induced electrical activity (the analogue of behavior) with a direct and temporally discrete application of dopamine (the analogue of reinforcement). After conditioning, the membrane properties of B51 are again modulated such that the cell is more likely to be active in the future. Such a highly reduced preparation is a promising candidate to study the mechanisms of dopamine-mediated reward and the conditioned expression of behavior at the level of the intracellular signaling cascades.

Conclusions

One of the important findings to emerge from recent studies on invertebrates is their capacity to exhibit various forms of associative learning. Of particular significance is the finding that at least some mollusks, such as Limax, exhibit higher-order features of classical conditioning, such as second-order conditioning and blocking. Contextual conditioning, conditioned discrimination learning, and contingency effects have been described in Aplysia (Colwill, Absher, and Roberts, 1988; Hawkins, Carew, and Kandel, 1986). Such higher-order features can be viewed in a cognitive context, and raise the interesting possibility that other complex behavioral phenomena will be identified as the capabilities of these animals are investigated further.

The possibility of relating cellular changes to complex behavior in invertebrates is encouraged by the progress that has already been made in examining the neural mechanisms of simple forms of nonassociative and associative learning. The results of these analyses of Aplysia have shown that: 1. learning involves changes in existing neural circuitry (one does not need the growth of new synapses and the formation of new circuits for learning and short-term memory to occur); 2. learning involves the activation of second messenger systems; 3. the second messengers affect multiple subcellular processes to alter the responsiveness of the neuron (at least one locus for the storage and readout of memory is the alteration of specific membrane currents); and 4. long-term memory requires new protein synthesis, whereas short-term memory does not.

While researchers have made considerable progress in the analysis of simple forms of learning in Aplysia, other invertebrates, and vertebrate model systems, there is still no complete mechanistic analysis available for any single example of simple learning. Many of the technical obstacles are being overcome, however, and it is likely that the analyses of several examples of learning will reach completion.

For the near future, major questions to be answered include the following: To what extent are mechanisms for classical and operant conditioning common both within any one species and between different species? What is the relationship between the initial induction of neuronal change (acquisition of learning) and the maintenance of the associative change? What are the relationships among different forms of learning, such as sensitization, classical conditioning, and operant conditioning?

Bibliography

Abrams, T. W., and Kandel, E. R. (1988). Is contiguity detection in classical conditioning a system or cellular property? Learning in Aplysia suggests a possible site. Trends in Neurosciences 11, 128-135.

Antonov, I., Antonova, I., Kandel, E. R., and Hawkins, R. D. (2001). The contribution of activity-dependent synaptic plasticity to classical conditioning in Aplysia. Journal of Neuroscience 21, 6,413-6,422.

Bao, J. X., Kandel, E. R., and Hawkins, R. D. (1998). Involvement of presynaptic and postsynaptic mechanisms in a cellular analog of classical conditioning at Aplysia sensory-motor neuron synapses in isolated cell culture. Journal of Neuroscience 18, 458-466.

Brembs, B., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., and Byrne J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science 296, 1,706-1,709.

Carew, T. J., Hawkins, R. D., and Kandel, E. R. (1983). Differential classical conditioning of a defensive withdrawal reflex in Aplysia californica. Science 219, 397-400.

Carew, T. J., Walters, E. T., and Kandel, E. R. (1981). Classical conditioning in a simple withdrawal reflex in Aplysia californica. Journal of Neuroscience 1, 1,426-1,437.

Colwill, R. M., Absher, R. A., and Roberts, M. L. (1988). Context-US learning in Aplysia californica. Journal of Neuroscience 8, 4,434-4,439.

Hawkins, R. D., Abrams, T. W., Carew, T. J., and Kandel, E. R.(1983). A cellular mechanism of classical conditioning in Aplysia: Activity-dependent amplification of presynaptic facilitation. Science 219, 400-405.

Hawkins, R. D., Carew, T. J., and Kandel, E. R. (1986). Effects of interstimulus interval and contingency on classical conditioning of the Aplysia siphon withdrawal reflex. Journal of Neuroscience 6, 1,095-1,701.

Lechner, H. A., Baxter, D. A., and Byrne, J. H. (2000). Classical conditioning of feeding in Aplysia: I. Behavioral analysis. Journal of Neuroscience 20, 3,369-3,376.

Lorenzetti, F. D., Baxter, D. A., and Byrne, J. H. (2000). Contingent reinforcement with dopamine modifies the properties of an individual neuron in Aplysia. Society for Neuroscience Abstracts 26, 1,524.

Murphy, G. G., and Glanzman, D. L. (1997). Mediation of classical conditioning in Aplysia californica by long-term potentiation of sensorimotor synapses. Science 278, 467-471.

Nargeot, R., Baxter, D. A., and Byrne J. H. (1999a). In vitro analog of operant conditioning in Aplysia: I. Contingent reinforcement modifies the functional dynamics of an identified neuron. Journal of Neuroscience 19, 2,247-2,260.

—— (1999b). In vitro analog of operant conditioning in Aplysia:II. Modifications of the functional dynamics of an identifiedneuron contribute to motor pattern selection. Journal of Neuroscience 19, 2,261-2,272.

Ocorr, K. A., Walters, E. T., and Byrne, J. H. (1985). Associative conditioning analog selectively increases cAMP levels of tail sensory neurons in Aplysia. Proceedings of the National Academy of Sciences of the United States of America 82, 2,548-2,552.

Schwartz, J. H., Bernier, L., Castellucci, V. F., Polazzolo, M., Saitoh, T., Stapleton, A., and Kandel, E. R. (1983). What molecular steps determine the time course of the memory for shortterm sensitization in Aplysia? Cold Spring Harbor Symposium on Quantitative Biology 48, 811-819.

Susswein, A. J., Schwarz, M., and Feldman, E. (1986). Learned changes of feeding behavior in Aplysia in response to edible and inedible foods. Journal of Neuroscience 6, 1,513-1,527.

Walters, E. T., and Byrne, J. H. (1983). Associative conditioning of single sensory neurons suggests a cellular mechanism for learning. Science 219, 405-408.

John H.Byrne

Revised byFred D.Lorenzetti

andJohn H.Byrne

Learning and Memory