Brain stimulation reward

History

In 1953, James Olds and Peter Milner observed that rats preferred to return to the region of the test apparatus where they received direct electrical stimulation to the septal area of the brain. From this demonstration, Olds and Milner inferred that the stimulation was rewarding, and through subsequent experiments, they confirmed that they could train rats to execute novel behaviors, such as lever pressing, in order to receive short pulse trains of brain stimulation. Olds and Milner discovered the reward mechanisms in the brain involved in positive reinforcement and their experiments led to the conclusion that electrical stimulation could serve as an operant reinforcer. According to B.F. Skinner, operant reinforcement occurs when a behavior is followed by the presentation of a stimulus, and it is considered essential to the learning of response habits. Their discovery enabled motivation and reinforcement to be understood in terms of their underlying physiology, and it led to further experimentation to determine the neural basis of reward and reinforcement. Since the initial discovery, the phenomenon of BSR has been demonstrated in all species tested, and Robert Heath similarly demonstrated that BSR can be applied to humans.

Brain stimulation reinforcement

Early studies on the motivational effects of brain stimulation addressed two primary questions: Which brain sites produce a rewarding effect when stimulated? and What drugs influence the response to stimulation and how? Investigation of the brain reward circuitry reveals that it consists of a distributed, multisynaptic circuit that determines both BSR and natural reward function. The natural drives that motivate and shape behavior reach the reward circuitry trans-synaptically through the peripheral senses of sight, sound, taste, smell, or touch. However, the laboratory induced rewards of intracranial electrical stimulation or drug injections directly activate the reward circuitry and bypass the peripheral sensory pathways. For this reason, electrical brain stimulation and drug injections provide a tool for identifying the reward circuitry within the central nervous system with some degree of anatomical and neurochemical specificity. Studies involving these two forms of laboratory reward showed stimulation of a broad range of limbic and diencephalic structures could be rewarding as well as implicated the dopamine-containing neurons of the mesolimbic dopamine system in motivational function. The motivational effect of intracranial self-stimulation varies substantially depending on the placement site of the surgically implanted electrode during electrical stimulation, and animals will work to stimulate different neural sites depending on their current state. Often, animals that work to initiate brain stimulation will also work to terminate the stimulation.

Measurement of brain stimulation reward

Since the initial demonstration of BSR by Olds and Milner, lever-pressing has been used as the traditional response motivated by stimulation, and experiments have established the rate of response as the dependent variable. The rewarding stimulus in BSR experiments is typically a train of short-duration pulses separated by interval pulses, which can be manipulated experimentally using the independent variables of stimulation intensity, pulse duration, or pulse frequency. The reward magnitude can be controlled as a function of stimulation intensity, and intensity is often used as the independent variable in studies to determine anatomical localization since high intensity stimulation is most likely to reach distal reward sites, whereas stimulation at low intensity is useful in localizing the boundaries of reward sites. Manipulation of stimulation frequency, the number of pulses per second, is preferable in pharmacological studies, which demonstrate an increase in response rate due to an increase in frequency. The independent variables of train and pulse duration can also be varied to determine the response rate of brain stimulation reward. Longer train durations produce more vigorous responding up to a point, after which rate of responding varies inversely with train length because animals begin to lever-press for additional stimulation before the previously earned train has finished.

Stimulation intensity, pulse duration, or pulse frequency can be varied to determine dose-response functions for BSR using the curve-shift paradigm, which generally resembles traditional pharmacological dose-response curves where the dose of stimulation rather than the dose of a drug is examined. This method allows the impact of a reward-altering treatment to be quantitatively estimated. In the curve-shift paradigm, low doses of stimulation fail to sustain the lever-press habit at a probability above chance. The response rates increase rapidly over a dynamic range of stimulation values as the stimulation dose increases, and once the high stimulation dose is reached, further increase in stimulation value does not yield a further increase in response rate over the subsequent static range. The response rate of brain stimulation reward in the dynamic range is sensitive to changes in the rewarding impact of stimulation, and changes in the rate of response over this range reflects changes in the reward magnitude . Rate-frequency, rate-intensity, or rate-duration functions make inferences about the potency and efficacy of stimulation, as well as elucidate how drugs alter the rewarding impact of stimulation. Analysis of the curve functions under experimental and control conditions determine whether the experimental treatment shifts the function. Experimental changes in the efficacy of stimulation that cause a shift up or down to a higher or lower asymptotic performance level reflect changes in the performance capabilities of the animals, whereas shifts to the right (to higher stimulation values) or to the left (to lower stimulation values) reflect synergism or antagonism of the rewarding impact of stimulation that determine its potency.

Relationship to natural rewards and drives

The relationship between induced laboratory and natural rewards (e.g. food, water and copulation) has long been debated, and much of the early research on BSR is focused on their similarities and differences. Experiments consistently indicate that BSR stimulates the reinforcement pathways normally activated by natural rewards, and drug reward or self-stimulation can exert more powerful activation of central reward mechanisms because they activate the reward circuit directly rather than through the peripheral nerves. BSR to the medial forebrain bundle (MFB) through either electrical or chemical means activates the neural pathways leading to natural drives. When specific regions of the hypothalamus are electrically stimulated, the stimulation causes eating, drinking, or copulation responses, and electrical stimulation is more reinforcing when the natural reward is available for consumption.

The difference between brain stimulation and natural rewards can be attributed to the lack of a deprivation state for brain stimulation that instinctively facilitates the drive to seek out brain stimulation. BSR also lacks the established neural representation in the animal’s memory that naturally facilitates the learning of reward expectancy. Both of these effects lead to diminished response rate for BSR in the early trials of a series; however, experiments have also shown that extinguished behavior can be quickly reinstated by a priming stimulation that refreshes the short-term association involved in reward expectancy. Experiments on BSR indicate that reinforcing brain stimulation may activate the natural pathways associated with natural drives as well as stimulate the reinforcement pathways that are usually activated by natural rewards.

Strength of drive

Rats will perform lever-pressing at rates of several thousand responses per hour for days in order to obtain direct electrical stimulation of the lateral hypothalamus. Multiple studies have demonstrated that rats will perform reinforced behaviors at the exclusion of all other behaviors. Experiments have shown rats to forgo food to the point of starvation in order to work for brain stimulation or intravenous cocaine when both food and stimulation are offered concurrently for a limited time each day. Rats will even cross electrified grids to press a lever, and they are willing to withstand higher levels of shock to obtain electrical stimulation than to obtain food .

Extinction

In comparison to behavior reinforced by natural reward, the reinforcing effects of self-stimulation through either electrodes or drug administration cause the behavior to be extinguished more rapidly and completely. Based on experiments with time spaced trials, Olds et al. speculated that the drive to self-stimulate is dependent on the immediate after-effects of the stimulation. Unlike brain stimulation, natural rewards provoke an innate or well-established cortical representation that can be associated with their accompanying sensory stimuli, causing them to produce expectancy-of-reward in subjects during subsequent trials. Since the electrical stimulation is an unnatural stimulus, the activating effect of the reward expectancy is diminished and therefore the behavior is subject to rapid extinction when the behavior is no longer reinforced.

Satiation

An interesting feature of BSR discovered during satiation experiments in rats is that reward itself does not produce satiety. Olds demonstrated that the lack of satiation associated with laboratory rewards allows animals to self-stimulation to sheer exhaustion and that satiation is dependent on the location of the electrical stimulation. In a 48-hour satiation test, rats with hypothalamic electrodes self-stimulated to exhaustion and showed no intrinsic satiation tendencies, whereas telencephalic electrodes showed radical slowing of self-stimulation after 4 to 8 hours.

Addiction

This evidence helps to elucidate why some behaviors become an addiction, a compulsive habit that is maintained despite harmful consequences. Laboratory rewards can establish compulsive self-administration habits of seeking and ingesting that qualify as addiction (1).Rats and monkeys have been shown to work in a compulsive manner to achieve intravenous injections of stimulants, and when access to the drugs is not limited, they will self-administer the drugs to the point of severe weight loss and death. This demonstrates that the habit of self-administration for intracranial brain stimulation or drug injections becomes compulsive almost immediately. BSR is hypothesized to be so effective in establishing compulsive habits because it bypasses the synaptic barriers in the sensory pathways to directly activate the reward pathway. The lack of delay in the reinforcement of a behavior motivated by BSR increases its effectiveness because even a delay of 1 second between the lever-press and the reward delivery can reduce its effectiveness . BSR offers insights into the neuroadaptations involved in reinforcement and addiction.

Anatomy of reward

Mapping and lesion studies on BSR were designed to determine the location of reward-relevant neurons as well as determine the signal pathways that are directly affected by brain stimulation. The site of intracranial self-stimulation leads to substantially different behavioral characteristics. Sites along the length of the medial forebrain bundle (MFB) through the lateral and posterior hypothalamus, the ventral tegmental area (VTA), and into the pons are associated with the strongest reward effects of stimulation.

Lateral hypothalamus

The lateral hypothalamus is a portion of the hypothalamus, and brain stimulation to this area of the reward system produces the highest response rates and subsequently the highest reward potency. Lesions in this region or along its boundary cause a loss of positive drive-reward behaviors as well as all other operant drive behaviors.

Medial forebrain bundle

The medial forebrain bundle (MFB) is the location of the most frequently investigated brain stimulation reward sites, and it is composed of a complex bundle of axons coming from the basal olfactory regions and the septal nuclei. MFB is not the sole anatomical substrate responsible for reinforcing brain stimulation; however, it is the main pathway for the ascending dopamine fibers and it functions to relay information from the VTA to the nucleus accumbens. The rewarding effect of the MFB depends on the ability of the stimulation to activate the mesocorticolimbic dopamine system.

Mesolimbic pathway

The mesolimbic pathway contains the VTA, nucleus accumbens, amygdalae, and the medial prefrontal cortex. The nucleus accumbens is a part of the striatum that integrates information from cortical and limbic brain structures to mediate behaviors the reinforce reward . The nucleus accumbens is a major target for the dopaminergic projections from the brainstem that are associated with BSR. The VTA is a group of neurons located close to the midline on the floor of the midbrain, and VTA is the origin of dopaminergic cell bodies that comprise the mesocorticolimbic dopamine system.

Modulation with drugs

When drugs are injected into the brain, they provide anatomical and neurochemical specific tools for investigating BSR. They are transported to all areas of the brain via the circulation system, and act on specific receptors that are restricted to particular classes of neurons. BSR and abusive drugs are hypothesized to activate the same brain mechanism and potentiate the rewarding effects of stimulation at the same brain sites. This is consistent with evidence that abusive drugs increase response in BSR by increasing the reward potency of the stimulation at the same brain sites and through the same mechanisms that makes the drugs rewarding. Studies using lesion, pharmacological, and anatomical mapping of the brain have revealed that many drugs of abuse (e.g. amphetamine, cocaine, opioids, nicotine, etc.) activate the reward circuitry of the MFB, which requires a stimulation to activate the mesocorticolimbic dopamine system. Current research focuses on the dopaminergic pathways as the substrate of brain stimulation reinforcement; however, both noradrenergic and dopaminergic pathways are important in brain stimulation reinforcement.

Dopamine reinforcement pathways

The mesolimbic pathway is activated trans-synaptically by normal rewards (food, water, copulation) but it can also be activated directly by the induced rewards of intravenous drugs or electrical or chemical brain stimulation. The mesolimbic pathway is one of the dopaminergic pathways in the brain that modulates behavioral responses to rewarding stimuli. It originates in the VTA and connects to the limbic system via the nucleus accumbens, the amygdala, and the medial prefrontal cortex. A number of drugs are rewarding when they are injected into the nucleus accumbens and act as mesolimbic dopamine terminals, and the axons of the mesolimbic dopamine system have high thresholds for stimulation .

Dopamine receptor antagonists and agonists

Drugs that alter the potency of BSR in either an antagonistic or synergistic manner are predominantly dopamine antagonists and agonists. The mechanism of synergism between addictive drugs and brain stimulation is the summation effect within or at the synapses of the mesolimbic system. Although dopamine plays a fundamental role in the rewarding effects of brain stimulation and abusive drugs, the exact nature of its role is still unclear, and since the reward circuitry is multisynaptic, dopamine is not the only reward transmitter. Dopamine neurons play an important role in incentive motivation and reinforcement associated with electrical stimulation to the MFB. Initially, stimulation of the MFB was thought to activate the dopamine fibers directly, it is now believed the stimulation at the MFB selectively activates afferent input into the dopamine system.

Cholinergic agents are rewarding when injected into the ventral tegmental area (VTA). Opiates including Mu and Delta opioids appear to have the strongest rewarding effects through afferents to the mesolimbic dopamine systems, and the presumed mechanism of action involves the disinhibition of the dopamine system by inhibition of nearby GABAergic neurons. Dopamine and opiate antagonists are competitive antagonists of BSR, and they impair the synaptic transmission of the reward signal either across the dopaminergic synapse or across a synapse in which dopamine has a modulatory function. On the other hand, drugs of abuse have been shown to synergize with BSR by reducing the BSR thresholds for amount of stimulation required as well as act to negate the effects of a reward antagonist. BSR synergistic drugs include: amphetamines, cocaine, heroin, morphine, nicotine, phenylcyclidine, mu and delta opioids, and cannabis. These synergistic drugs summate with BSR and serve as rewards themselves by elevating the dopamine concentration at forebrain dopamine nerve terminals, including the nucleus accumbens.

Clinical relevance

Mechanisms of BSR offer a tool that provides insight into the way the brain governs behavior through motivation and reinforcement, especially in regards to addictive behavior. Further studies will elucidate the mechanisms by which addictive drugs initiate their habit-forming behavior as well as enhance knowledge of the neuroadaptations that occurs within the brain as a result of electrical stimulation or chemical injection. BSR will continue to shed light on and lead to the development of therapies to treat motivational compulsions such as compulsive eating, drinking, drug use, gambling, etc. Advances in technology and the development of an intracranial stimulation implant will assist patients suffering from disorders such as obsessive-compulsive disorder and could be used to treat a variety of psychiatric conditions, including depression, by modulating the activity of the reward system.