Dopaminergic systems create reward seeking despite adverse consequences.

Resource-seeking behaviours are ordinarily constrained by physiological needs and threats of danger, and the loss of these controls is associated with pathological reward seeking1. Although dysfunction of the dopaminergic valuation system of the brain is known to contribute towards unconstrained reward seeking2,3, the underlying reasons for this behaviour are unclear. Here we describe dopaminergic neural mechanisms that produce reward seeking despite adverse consequences in Drosophila melanogaster. Odours paired with optogenetic activation of a defined subset of reward-encoding dopaminergic neurons become cues that starved flies seek while neglecting food and enduring electric shock punishment. Unconstrained seeking of reward is not observed after learning with sugar or synthetic engagement of other dopaminergic neuron populations. Antagonism between reward-encoding and punishment-encoding dopaminergic neurons accounts for the perseverance of reward seeking despite punishment, whereas synthetic engagement of the reward-encoding dopaminergic neurons also impairs the ordinary need-dependent dopaminergic valuation of available food. Connectome analyses reveal that the population of reward-encoding dopaminergic neurons receives highly heterogeneous input, consistent with parallel representation of diverse rewards, and recordings demonstrate state-specific gating and satiety-related signals. We propose that a similar dopaminergic valuation system dysfunction is likely to contribute to maladaptive seeking of rewards by mammals.

The long noncoding RNA mimi scaffolds neuronal granules to maintain nervous system maturity.

RNA binding proteins and messenger RNAs (mRNAs) assemble into ribonucleoprotein granules that regulate mRNA trafficking, local translation, and turnover. The dysregulation of RNA-protein condensation disturbs synaptic plasticity and neuron survival and has been widely associated with human neurological disease. Neuronal granules are thought to condense around particular proteins that dictate the identity and composition of each granule type. Here, we show in Drosophila that a previously uncharacterized long noncoding RNA, mimi, is required to scaffold large neuronal granules in the adult nervous system. Neuronal ELAV-like proteins directly bind mimi and mediate granule assembly, while Staufen maintains condensate integrity. mimi granules contain mRNAs and proteins involved in synaptic processes; granule loss in mimi mutant flies impairs nervous system maturity and neuropeptide-mediated signaling and causes phenotypes of neurodegeneration. Our work reports an architectural RNA for a neuronal granule and provides a handle to interrogate functions of a condensate independently of those of its constituent proteins.

Prior experience conditionally inhibits the expression of new learning in Drosophila.

Prior experience of a stimulus can inhibit subsequent acquisition or expression of a learned association of that stimulus. However, the neuronal manifestations of this learning effect, named latent inhibition (LI), are poorly understood. Here, we show that prior odor exposure can produce context-dependent LI of later appetitive olfactory memory performance in Drosophila. Odor pre-exposure forms a short-lived aversive memory whose lone expression lacks context-dependence. Acquisition of odor pre-exposure memory requires aversively reinforcing dopaminergic neurons that innervate two mushroom body compartments-one group of which exhibits increasing activity with successive odor experience. Odor-specific responses of the corresponding mushroom body output neurons are suppressed, and their output is necessary for expression of both pre-exposure memory and LI of appetitive memory. Therefore, odor pre-exposure attaches negative valence to the odor itself, and LI of appetitive memory results from a temporary and context-dependent retrieval deficit imposed by competition with the parallel short-lived aversive memory.

Changing memories on the fly: the neural circuits of memory re-evaluation in Drosophila melanogaster.

Associative learning leads to modifications in neural networks to assign valence to sensory cues. These changes not only allow the expression of learned behavior but also modulate subsequent learning events. In the brain of the adult fruit fly, Drosophila melanogaster, olfactory memories are established as dopamine-driven plasticity in the output of a highly recurrent network, the mushroom body. Recent findings have highlighted how these changes in the network can steer the strengthening, weakening and formation of parallel memories when flies are exposed to subsequent training trials, conflicting situations or the reversal of contingencies. Together, these processes provide an initial understanding of how learned information can be used to guide the re-evaluation of memories.

Switching Gears, Structuring the Right Search Strategy.

Nematodes can use local and global search strategies to find food. In this issue of Neuron, López-Cruz et al. (2019) unravel a neural circuit mechanism that allows worms to select and switch between these search modes depending on recent experience of food.

Integration of Parallel Opposing Memories Underlies Memory Extinction.

Accurately predicting an outcome requires that animals learn supporting and conflicting evidence from sequential experience. In mammals and invertebrates, learned fear responses can be suppressed by experiencing predictive cues without punishment, a process called memory extinction. Here, we show that extinction of aversive memories in Drosophila requires specific dopaminergic neurons, which indicate that omission of punishment is remembered as a positive experience. Functional imaging revealed co-existence of intracellular calcium traces in different places in the mushroom body output neuron network for both the original aversive memory and a new appetitive extinction memory. Light and ultrastructural anatomy are consistent with parallel competing memories being combined within mushroom body output neurons that direct avoidance. Indeed, extinction-evoked plasticity in a pair of these neurons neutralizes the potentiated odor response imposed in the network by aversive learning. Therefore, flies track the accuracy of learned expectations by accumulating and integrating memories of conflicting events.

Do the right thing: neural network mechanisms of memory formation, expression and update in Drosophila.

When animals learn, plasticity in brain networks that respond to specific cues results in a change in the behavior that these cues elicit. Individual network components in the mushroom bodies of the fruit fly Drosophila melanogaster represent cues, learning signals and behavioral outcomes of learned experience. Recent findings have highlighted the importance of dopamine-driven plasticity and activity in feedback and feedforward connections, between various elements of the mushroom body neural network. These computational motifs have been shown to be crucial for long term olfactory memory consolidation, integration of internal states, re-evaluation and updating of learned information. The often recurrent circuit anatomy and a prolonged requirement for activity in parts of these underlying networks, suggest that self-sustained and precisely timed activity is a fundamental feature of network computations in the insect brain. Together these processes allow flies to continuously adjust the content of their learned knowledge and direct their behavior in a way that best represents learned expectations and serves their most pressing current needs.

Re-evaluation of learned information in Drosophila.

Animals constantly assess the reliability of learned information to optimize their behaviour. On retrieval, consolidated long-term memory can be neutralized by extinction if the learned prediction was inaccurate. Alternatively, retrieved memory can be maintained, following a period of reconsolidation during which it is labile. Although extinction and reconsolidation provide opportunities to alleviate problematic human memories, we lack a detailed mechanistic understanding of memory updating. Here we identify neural operations underpinning the re-evaluation of memory in Drosophila. Reactivation of reward-reinforced olfactory memory can lead to either extinction or reconsolidation, depending on prediction accuracy. Each process recruits activity in specific parts of the mushroom body output network and distinct subsets of reinforcing dopaminergic neurons. Memory extinction requires output neurons with dendrites in the α and α' lobes of the mushroom body, which drive negatively reinforcing dopaminergic neurons that innervate neighbouring zones. The aversive valence of these new extinction memories neutralizes previously learned odour preference. Memory reconsolidation requires the γ2α'1 mushroom body output neurons. This pathway recruits negatively reinforcing dopaminergic neurons innervating the same compartment and re-engages positively reinforcing dopaminergic neurons to reconsolidate the original reward memory. These data establish that recurrent and hierarchical connectivity between mushroom body output neurons and dopaminergic neurons enables memory re-evaluation driven by reward-prediction error.

Memory-Relevant Mushroom Body Output Synapses Are Cholinergic.

Memories are stored in the fan-out fan-in neural architectures of the mammalian cerebellum and hippocampus and the insect mushroom bodies. However, whereas key plasticity occurs at glutamatergic synapses in mammals, the neurochemistry of the memory-storing mushroom body Kenyon cell output synapses is unknown. Here we demonstrate a role for acetylcholine (ACh) in Drosophila. Kenyon cells express the ACh-processing proteins ChAT and VAChT, and reducing their expression impairs learned olfactory-driven behavior. Local ACh application, or direct Kenyon cell activation, evokes activity in mushroom body output neurons (MBONs). MBON activation depends on VAChT expression in Kenyon cells and is blocked by ACh receptor antagonism. Furthermore, reducing nicotinic ACh receptor subunit expression in MBONs compromises odor-evoked activation and redirects odor-driven behavior. Lastly, peptidergic corelease enhances ACh-evoked responses in MBONs, suggesting an interaction between the fast- and slow-acting transmitters. Therefore, olfactory memories in Drosophila are likely stored as plasticity of cholinergic synapses.

Differences in long-term memory stability and AmCREB level between forward and backward conditioned honeybees (Apis mellifera).

In classical conditioning a predictive relationship between a neutral stimulus (conditioned stimulus; CS) and a meaningful stimulus (unconditioned stimulus; US) is learned when the CS precedes the US. In backward conditioning the sequence of the stimuli is reversed. In this situation animals might learn that the CS signals the end or the absence of the US. In honeybees 30 min and 24 h following backward conditioning a memory for the excitatory and inhibitory properties of the CS could be retrieved, but it remains unclear whether a late long-term memory is formed that can be retrieved 72 h following backward conditioning. Here we examine this question by studying late long-term memory formation in forward and backward conditioning of the proboscis extension response (PER). We report a difference in the stability of memory formed upon forward and backward conditioning with the same number of conditioning trials. We demonstrate a transcription-dependent memory 72 h after forward conditioning but do not observe a 72 h memory after backward conditioning. Moreover we find that protein degradation is differentially involved in memory formation following these two conditioning protocols. We report differences in the level of a transcription factor, the cAMP response element binding protein (CREB) known to induce transcription underlying long-term memory formation, following forward and backward conditioning. Our results suggest that these alterations in CREB levels might be regulated by the proteasome. We propose that the differences observed are due to the sequence of stimulus presentation between forward and backward conditioning and not to differences in the strength of the association of both stimuli.

Activity of defined mushroom body output neurons underlies learned olfactory behavior in Drosophila.

During olfactory learning in fruit flies, dopaminergic neurons assign value to odor representations in the mushroom body Kenyon cells. Here we identify a class of downstream glutamatergic mushroom body output neurons (MBONs) called M4/6, or MBON-ß2ß'2a, MBON-ß'2mp, and MBON-γ5ß'2a, whose dendritic fields overlap with dopaminergic neuron projections in the tips of the ß, ß', and γ lobes. This anatomy and their odor tuning suggests that M4/6 neurons pool odor-driven Kenyon cell synaptic outputs. Like that of mushroom body neurons, M4/6 output is required for expression of appetitive and aversive memory performance. Moreover, appetitive and aversive olfactory conditioning bidirectionally alters the relative odor-drive of M4ß' neurons (MBON-ß'2mp). Direct block of M4/6 neurons in naive flies mimics appetitive conditioning, being sufficient to convert odor-driven avoidance into approach, while optogenetically activating these neurons induces avoidance behavior. We therefore propose that drive to the M4/6 neurons reflects odor-directed behavioral choice.

Two inhibitors of the ubiquitin proteasome system enhance long-term memory formation upon olfactory conditioning in the honeybee (Apis mellifera).

In honeybees (Apis mellifera), the proteasome inhibitor Z-Leu-Leu-Leu-CHO (MG132) enhances long-term memory (LTM) formation. Studies in vertebrates using different inhibitors of the proteasome demonstrate the opposite, namely an inhibition of memory formation. The reason for this contradiction remains unclear. MG132 is an inhibitor of the proteasome, but also blocks other proteases. Accordingly, one possible explanation might be that other proteases affected by MG132 are responsible for the enhancement of LTM formation. We test this hypothesis by comparing the effect of MG132 and the more specific proteasome inhibitor clasto-lactacystin beta-lactone (ß-lactone). We show that these two inhibitors block the activity of the proteasome in honeybee brains to a similar extent, do not affect the animals' survival but do enhance LTM retention upon olfactory conditioning. Thus, the enhancement of LTM formation is not due to MG132-specific side effects, but to inhibition of a protease targeted by MG132 and ß-lactone, i.e. the proteasome.

Short- and long-term memories formed upon backward conditioning in honeybees (Apis mellifera).

In classical conditioning, the temporal sequence of stimulus presentations is critical for the association between the conditioned stimulus (CS) and the unconditioned stimulus (US). In forward conditioning, the CS precedes the US and is learned as a predictor for the US. Thus it acquires properties to elicit a behavioral response, defined as excitatory properties. In backward conditioning, the US precedes the CS. The CS might be learned as a predictor for the cessation of the US acquiring inhibitory properties that inhibit a behavioral response. Interestingly, behavior after backward conditioning is controlled by both excitatory and inhibitory properties of the CS, but the underlying mechanisms determining which of these opposing properties control behavior upon retrieval is poorly understood. We performed conditioning experiments in the honeybee (Apis mellifera) to investigate the CS properties that control behavior at different time points after backward conditioning. The CS properties, as characterized by the retardation or enhancement of subsequent acquisition, were examined 30 min and 24 h after backward conditioning. We found that 30 min after backward conditioning, the CS acquired an inhibitory property during backward conditioning depending on the intertrial interval, the number of trials, and the odor used as the CS. One day after backward conditioning, we observed significant retardation of acquisition. In addition, we demonstrated an enhanced, generalized odor response in the backward conditioned group compared to untreated animals. These results indicate that two long-lasting opposing memories have been formed in parallel: one about the excitatory properties of the CS and one about the inhibitory properties of the CS.

Reinstatement in honeybees is context-dependent.

During extinction animals experience that the previously learned association between a conditioned stimulus (CS) and an unconditioned stimulus (US) no longer holds true. Accordingly, the conditioned response (CR) to the CS decreases. This decrease of the CR can be reversed by presentation of the US alone following extinction, a phenomenon termed reinstatement. Reinstatement and two additional phenomena, spontaneous recovery and renewal, indicate that the original CS-US association is not lost through extinction but can be reactivated through different processes. In honeybees (Apis mellifera), spontaneous recovery, i.e., the time-dependent return of the CR, has been demonstrated, suggesting that also in these insects the original CS-US association is not lost during extinction. To support this notion, we ask whether honeybees show reinstatement after extinction. In vertebrates reinstatement is context-dependent, so we examined whether the same holds true for honeybees. We demonstrate reinstatement in restrained honeybees and show that reinstatement is context-dependent. Furthermore, we show that an alteration of the color of light illuminating the experimental setup suffices to indicate a contextual change. We conclude that in honeybees the initially formed CS-US memory is not lost after extinction. Rather, honeybees might learn about the context during extinction. This enables them to adequately retrieve one of the two opposing memories about the CS that have been formed after extinction.

A role of protein degradation in memory consolidation after initial learning and extinction learning in the honeybee (Apis mellifera).

Protein degradation is known to affect memory formation after extinction learning. We demonstrate here that an inhibitor of protein degradation, MG132, interferes with memory formation after extinction learning in a classical appetitive conditioning paradigm. In addition, we find an enhancement of memory formation when the same inhibitor is applied after initial learning. This result supports the idea that MG132 targets an ongoing consolidation process. Furthermore, we demonstrate that the sensitivity of memory formation after initial learning and extinction learning to MG132 depends in the same way on the number of CS-US trials and the intertrial interval applied during initial learning. This supports the idea that the learning parameters during acquisition are critical for memory formation after extinction and that protein degradation in both learning processes might be functionally linked.

Average group behavior does not represent individual behavior in classical conditioning of the honeybee.

Conditioned behavior as observed during classical conditioning in a group of identically treated animals provides insights into the physiological process of learning and memory formation. However, several studies in vertebrates found a remarkable difference between the group-average behavioral performance and the behavioral characteristics of individual animals. Here, we analyzed a large number of data (1640 animals) on olfactory conditioning in the honeybee (Apis mellifera). The data acquired during absolute and differential classical conditioning differed with respect to the number of conditioning trials, the conditioned odors, the intertrial intervals, and the time of retention tests. We further investigated data in which animals were tested for spontaneous recovery from extinction. In all data sets we found that the gradually increasing group-average learning curve did not adequately represent the behavior of individual animals. Individual behavior was characterized by a rapid and stable acquisition of the conditioned response (CR), as well as by a rapid and stable cessation of the CR following unrewarded stimuli. In addition, we present and evaluate different model hypotheses on how honeybees form associations during classical conditioning by implementing a gradual learning process on the one hand and an all-or-none learning process on the other hand. In summary, our findings advise that individual behavior should be recognized as a meaningful predictor for the internal state of a honeybee--irrespective of the group-average behavioral performance.

Behavioural pharmacology in classical conditioning of the proboscis extension response in honeybees (Apis mellifera).

Honeybees (Apis mellifera) are well known for their communication and orientation skills and for their impressive learning capability(1,2). Because the survival of a honeybee colony depends on the exploitation of food sources, forager bees learn and memorize variable flower sites as well as their profitability. Forager bees can be easily trained in natural settings where they forage at a feeding site and learn the related signals such as odor or color. Appetitive associative learning can also be studied under controlled conditions in the laboratory by conditioning the proboscis extension response (PER) of individually harnessed honeybees(3,4). This learning paradigm enables the study of the neuronal and molecular mechanisms that underlie learning and memory formation in a simple and highly reliable way(5-12). A behavioral pharmacology approach is used to study molecular mechanisms. Drugs are injected systemically to interfere with the function of specific molecules during or after learning and memory formation(13-16). Here we demonstrate how to train harnessed honeybees in PER conditioning and how to apply drugs systemically by injection into the bee flight muscle.