47: Striatum as a mosaic of broken mirrors

On September 18, 2025 By Scott FergusonLeave a comment

The mosaic of broken mirrors is an analogy for the striatum in the basal ganglia [Da Cunha et al 2009]. The striatum represents a vertebrate’s actions and environment in a broken and overlapping fashion. While actions have focal projections to the striatum, the contextual input is broad and diffuse [Fee MS 2012]. While the hippocampus represents the environment globally, the striatum depends on piecewise representation. This means striatal learning is unable to generalize, because mosaic fragments lack a global perspective [Da Cunha et al 2009].

In the essays I’ve used the striatum as a timeout for food seek from an odor plume. When a food odor doesn’t have any food, or the odor is behind a barrier, the animal needs some sort of timeout to give up seeking the odor, turn away from the false odor, and search else where. However, the current simulation only uses the seek action for the timeout; it lacks environment context.

A simple illustration depicting two circular shapes; one has a blue and red figure inside, while the other contains a green star-like shape. — *Simulation screenshot showing the model animal trapped into perseverating seek in the center of a false odor plume. Circles represent odor plumes and the star represents food.*

The above simulation screenshot shows the general problem. The circles represent food odor plumes and the star represents food. The animal has followed a false odor plume and will continue circling the center until the striatum timeout. However, there is a nearby valid odor plume with food in it. The animal should avoid the false odor plume and search the correct odor plume, but currently it can’t distinguish the two, because it’s only using its own seek action as a key. If the animal could detect environmental context differences between the two plumes, it could search more effectively.

As a context, odor neighborhoods [Jacobs 2022], [Marin et al 2021] can represent a primitive representation of place. If each place has a different set of odor molecules, the animal can use that odor scene to distinguish false odor plumes from true food odor sources.

Striatum as timeout

In the essays I’ve used the striatum as a timeout mechanism to prevent perseveration, specifically the S.d2 (striatum projection neurons with D2.i Gi inhibiting dopamine receptor), which use A2a.s (adenosine Gs stimulating receptor) to measure the buildup of Ado (adenosine). The striatum’s projection neurons are roughly evenly divided between S.d1 (D1.s Gs stimulating dopamine receptor) and S.d2. For long stimulations, S.d1 generally motivate the current action and S.d2 opposes the action and produces avoidance [Soares-Cunha et al 2020], but for short stimulations both are active for action initiation [Cui G et al 2013].

Ado signaling limits swimming in frog tadpoles [Dale 1998]. In the striatum A2a.s receptors in S.d2 projection neurons also detect Ado buildup from neuron activity and from astrocytes that monitor neuron activity [Kang S et al 2020]. The Ado buildup activates S.d2 neurons and increases its internal PKA (protein kinase A) [Ma L et al 2022]. PKG slowly builds up during activity with a buildup time constant on the order of 10-20s and a decay constant on the order of 70s [Ma L et al 2022], but the increase appears to be log or sigmoid-like, not linear, suggesting that longer timeouts would be possible with high thresholds or opposition from S.d1. In S.d2, the PKA buildup enables the release of the opioid enkephalin [Konradi et al 2003], [Hook et al 2008], which activates the DOR.i (δ-opioid inhibitory receptor), which is necessary for the inhibitory/avoidance behavior for S.d2 [Soares-Cunha et al 2020].

Because the essay simulation needs a timeout to limit food seek perseveration, and the Ado and S.d2 avoidance chain could plausibly implement that timeout, I’ve been using it as the basis for the simulation’s timeout. This function seems evolutionarily plausible, because avoiding perseveration is important to keep the animal from unproductive seeking, and the implementation is fairly straightforward, only requiring already existing Ado timeout sensing and S.d2, without needing the entire basal ganglia. However, up to this point, that timeout has only used the action as a key, and has not included any context. Essay 15 and 16 did cover odor seek timeout in the context of associative habituation, but on the context of the fruit fly mushroom body.

Action and context as striatum inputs

S.pn (striatum projection neurons: both S.d1 and S.d2) are often called medium spiny neurons because their extensive dendrites are covered with spines. Spines are small dendrite compartments that receive axon inputs. Spines can compartmentalize Ca2+ (calcium) transients [Fang LZ and Creed 2024], meaning S.pn activation is not necessarily global across the entire dendrite tree, but compartmentalized. In S.d (dorsal striatum), cortical axons attach to S.pn spines, while T.pf (parafascicular thalamus) connects to the dendrite shaft [Fee MS 2012]. T.pf signals include ongoing action feedback and efference copies from the hindbrain and midbrain, including OT (optic tectum), while the cortex provides environmental context.

Diagram illustrating the connection between cortical inputs (C), thalamic inputs (T.pf), and striatum projection neurons (S.pn) via dendritic spines. — Rough diagram of inputs to S.pn dendrites. Cortical input is to distal dendrites and spines, while T.pf is more proximal and to dendrite shaft. C (cortex), S.pn (striatum projection neuron), T.pf (parafascicular thalamus).

Songbirds have a portion of the basal ganglia devoted to singing called Area X [Kornfeld et al 2020]. Area X receives song action variability information from C.lman (lateral nucleus of anterior nidopallum) and timing context from C.hvc, which are areas of the songbird cortex. C.lman provides an action efference copy of the variation actions [Fee MS 2014]. The majority (85%) of C.hvc input is on S.pn spines, while 55% of the C.lman action information is on the dendrite shafts [Kornfeld et al 2020]. The action efferent copy input to S.pn are not plastic, while the contextual input on the shafts is plastic [Fee MS 2012]. The context drives S.pn activation to an Up state, gating the core action driver [Fee MS 2012]. Action input to the striatum is focuses, while contextual information is diffuse [Fee MS 2012].

S.pn inputs can differ in their attachment to spines or dendrite shafts, and they can also differ in triggering behavior and for Up states. S.pn are normally hyper polarized, meaning they are normally especially difficult to trigger. Some inputs can shift S.pn into an Up state, where they are more easily triggered. In S.v (ventral striatum), inputs from E.sub.v (ventral subiculum in the hippocampal complex) can shift S.pn into an Up state for hundreds of millisecond [O’Donnell and Grace 1995], [Sesack and Grace 2010]. When E.sub.v is disable, S.v spontaneous or bistable activity halts, and other inputs such as F.pfc (prefrontal cortex) can’t trigger action potentials [O’Donnell and Grace 1995]. Up state transitions can be facilitated by dopamine [Fang LZ and Creed 2024], [Lahiri and Bevan 2020] and astrocyte sensing of glutamate activity [D’Ascenzo et al 2007], [Yu X et al 2018].

For the purposes of the essay simulation, these differences in striatum input suggest it’s plausible to treat action input and contextual input as distinct types of input, following [Fee MS 2012]. Specifically, that the combination of an action input and a context is required to drive the striatum timeout. This means that a seek timeout can be specific to a context and not overflow to other contexts.

Innate and contextual odors

In vertebrates, O.sn (odor sensory neurons) axons project to O.gl (glomerules) in Ob (olfactory bulb), where they connect with O.pn (odor projection neuron: mitral and tufted cells in mammals) dendrites. Each O.gl is a large neuropil (axon and dendrite connection area) where multiple O.sn and O.pn combine. In mammals, each O.gn responds to a single O.sn odor feature. Each O.gl typically responds to several odor molecules, and each odor molecule drives multiple O.gl [Wilson and Mainen 2006], [Weiss 2020]. In other vertebrates, each O.gl can combine inputs from multiple O.sn. Insect odor processing also uses glomerules, but this shared structure is independent evolution not homology because even the underlying odor detection receptors are unrelated between insects and vertebrates [Weiss 2020]. The glomeruli structure is likely simply an effective way of connecting multiple O.sn to O.pn.

As an analogy that the simulation uses, consider phonemes in a syllable, where each syllable is like an odor molecule and each phoneme is like a glomeruli. The syllable “cat” consists of “c-“, “-a-“, and “-t”, corresponding to three glomerules, and “c-” is driven by many different syllables. So, O.gl doesn’t identify the whole odor, but only a feature of the odor, like “c-“, but the features can be recombined to identify the odor.

The odor glomerules in vertebrates are divided into a smaller innate group and a larger contextual group. Most mammals have distinct Ob and O.a (accessory olfactory bulb). The lamprey Ob.m (medial Ob) projects directly to the midbrain, including Hb.m (medial habenula) and V.pt (posterior tuberculum) [Derjean et al 2010], [Beauséjour et al 2022], while the Ob.l (lateral Ob) projects to Pa (pallium/cortex) and basal ganglia [Beauséjour et al 2022], [Beauséjour et al 2024], [Suryanarayana et al 2021].

Previous essays have only used the innate Ob.m projection and ignored the Ob.l projection. This essays adds the Ob.l context projection to S.o t(olfactory tubercle), which is a part of S.v (ventral striatum) with large, direct olfactory input, and output to H.l (lateral hypothalamus) and Pv (ventral pallidum). The Ob.l context may represent odor neighborhoods, introducing a notion of place.

Odor neighborhoods

Odors rarely occur in isolation, are dynamic in space and time [Marin et al 2021], and form spatial neighborhoods [Jacobs 2022]. Olfactory curs influence E.hc (hippocampus) place fields, and place cells in blind rats are similar to sighted rats [Marin et al 2021]. In O.pir.p (posterior piriform cortex/olfactory cortex), place can be decoded to 90% accuracy with 240 neurons [Poo C et al 2022]. The olfactory spatial hypothesis considers odor are more for navigation than for identification [Jacobs 2012], where an odor neighborhood is a local area of odor mixtures.

It seems plausible that an early proto-vertebrate could use a combination of odor features from O.gl in an early S.v to restrict a seek timeout to a local place. The circuit is a straightforward extension of existing Ado timeout circuitry.

Seek striatum

For the essay’s simulation, I’m using odor context from Ob.l as a neighborhood detector. The striatum has two inputs: an action that enables the striatum during a seek and a place context identified by odor to restrict the search.

A simulation screenshot depicting various spatial representations of sensory inputs, including graphs and spatial maps related to odor detection and processing mechanisms. — Screenshot of the seek task blocked by a U-shaped barrier with odor neighborhoods represented by color and pattern. The local odor “rat” is represented by odor glomerules for “r-“, “-a-“, and “-t”.

The above screenshot shows the animal after its timeout from a failed odor seek when blocked by a barrier. The star represents food and the circle is an odor plume. Each pattern in the arena represents an odor neighborhood. The right side of the screenshot shows the active glomerules for the neighborhood, represented by “r-“, “-a-” and “-t”. I’m using syllables to represent odor molecules and phonemes to represent odor features detected by Ob glomerules.

Fruit fly mushroom body and Kenyon cells

Because the hypothetical proto-vertebrate would have a much simpler striatum than the mammal striatum, consider the comparison with the fruit fly MB (mushroom body) and its KC (Kenyon cells), which has a similar structure to the Sv projections to Pv, but has a much smaller scale. The mushroom body is highly conserved among insects and possibly predates all arthropods [Fiala and Kaun 2024], and serves as an odor pattern detector. In fruit flies, 52 O.pn project to ~2000 KC [Chan ICW et al 2024], which project to 24 MBON (mushroom body output neurons) [Seki et al 2017]. Each KC has three to seven claws [Zheng et al 2022], which are essentially single-connection dendrites.

A diagram illustrating the connectivity of Kenyon cells (KCs) in the fruit fly mushroom body, showing olfactory sensory neuron (OSN) inputs, projection neurons (PNs), and connections to mushroom body output neurons (MBONs) with their neurotransmitter types. — Architecture of the Drosophila mushroom body, adapted from [Aso Y et al 2014]. For this essay, only the left side projections of PN to KC are important. KC (Kenyon cell), PN (olfactory projection neuron), OSN (olfactory sensory neuron)

The above diagram shows the fruit fly mushroom body, but only the KC on the left are relevant for this essay. The MBON on the right would correspond to Pv (ventral pallidum) in this analogy. Each of the 2000 KC receive essentially random olfactory input from the O.pn, where each KC receives 3-7 O.pn inputs.

This mushroom body structure roughly corresponds to vertebrate O.pn projections to S.ot (ventral striatum olfactory tubercle), which projects to Pv. Although the connectivity pattern is similar, the two structures are not homologous in any fashion. Insect O.sn and vertebrate O.sn use entirely separate olfactory receptor families, and KCs use ACh (acetylcholine) as a neurotransmitter, while S.pn use GABA and vertebrate-specific opioids. The point of the analogy here is only to compare the scale of the O.gl and S.ot for a possible proto-vertebrate because the mammal S.ot is vastly too large to be plausible for that ancestor.

The MB has been compared to vertebrate CB-like (cerebellum-like) structures in the hindbrain [Farris 2011], suggesting that both serve as adaptive sensory filters. CB-like structures have dual inputs: one is sensory-specific input and the other is multimodal contextual. The MB also serves as a brake on insect locomotion, because fruit flies with MB lesions are less likely to stop locomotion once begun moving. This locomotion stopping is similar to the seek perseveration timeout in this essay.

Diagram comparing the olfactory processing pathways in insects and vertebrates. The insect mushroom body pathway includes odor sensory neurons (O.sn), odor projection neurons (O.pn), Kenyon cells (KC), and mushroom body output neurons (MBON), leading to seek/avoid responses. The vertebrate pathway similarly includes O.sn, O.pn, striatal projection neurons (S.pn), and the ventral pallidum (Pv), also leading to seek/avoid responses. — Analogy between the insect mushroom body and the vertebrate ventral basal ganglia. KC (Kenyon cells), MBON (mushroom body output neuron), O.pn (olfactory projection neuron), O.sn (olfactory sensory neuron), Pv (ventral pallidum), S.pn (striatum projection neuron)

For scale, consider using the lamprey Ob during the syllable analogy. The lamprey has approximately 40 olfactory receptor genes [Beauséjour et al 2020]. If we exclude about 10 innate from Ob.m, the 30 are contextual odors in Ob.l Consider splitting each odor as a syllable into three odor features as pheromones. If the 30 lamprey O.gl were organized like phonemes, then it might have 10 initial consonants, 10 vowels, and 10 final consonants. Suppose each S.pn receives three inputs from O.pn: one of 10 initial consonants, one of 10 vowels, and one of 10 final consonant. The 1000 S.pn would cover the possible syllables, expanding the dimensionality from 30 odor phonemes to 1000 odor syllables. Analogously, the Drosophila 52 O.pn phonemes expand to ~2000 KC syllables, roughly the same order of magnitude. Of course, the olfactory neighborhoods aren’t actually nicely ordered into convenient human-readable syllables, but it’s a convenience analogy, particularly for the simulation.

Returning to the original metaphor of the mosaic of broken mirrors [Da Cunha et al 2009], the O.pn breaks odor molecules (syllables and unbroken mirror) into a broken set of odor features (phonemes and mosaic tessera), and randomly reassembles the features in S.pn like tessera in a mosaic, partially recovering the original syllable structure, albeit lossy. Because the features are broken pieces stripped from their original odor molecule identity, the system could add other modalities, such as lateral line or whisker sensing, or temperature, or color fragments, before combining them, Although the fruit fly KCs primarily combine odor inputs, they also include a smaller number of non-odor inputs such as visual, gustatory, mechanosensory, and proprioceptive inputs [Farris 2011].

S.core seek and S.msh roam

So far I’ve used the striatum as a timeout to avoid seek perseveration, giving up on a failed food odor. Adding an odor neighborhood context improves the accuracy of seek perseveration control. Now that odor neighborhoods are available, the animal could also avoid neighborhoods that it’s already searched, essentially creating a memory breadcrumb, as explored in essay 44.

A diagram showcasing a grid with labeled sections including 'dm,' 'dl,' 'msh.d,' 'msh.v,' 'core,' 'lsh,' 'mot,' and 'lot,' indicating different components or regions. — Rough topographic divisions of the striatum. The blue S.msh.d is used for roam, and the amber S.core and S.lsh are used for seek. S.core (Sv core), S.dl (dorsolateral striatum), S.dm (dorsomedial striatum), S.lot (lateral S.ot – olfactory tubercle of Sv), S.lsh (lateral shell of Sv), S.mot (medial S.ot), S.msh.d (medial shell of Sv, dorsal part), S.msh.v (medial shell of Sv, ventral part), Sv (ventral striatum aka nucleus accumbens)

Sv (ventral striatum) is divided into S.core (Sv core) and S.sh (S shell), where S.core surrounds the anterior commissure, which connects Ob and amygdala. The shell further divides into S.msh (medial shell) and S.lsh (lateral shell), with S.ot also dividing into S.mot (medial S.ot) and S.lot (lateral S.lot). S.msh itself divides into S.msh.d (dorsal S.msh) and S.msh.v (ventral S.msh). These regions have distinct genetic transcription factor types and connectivity, and S.msh may be even more complicated with further genetically defined subtypes [Chen R et al 2021].

Functionally, S.lsh and S.core are more similar, related to cues and seek [Floresco 2015], [Chen G et al 2023], [Ding YD et al 2022], [Dobrovitsky 2017], while S.msh is distinct and related to place [Al-Hasani et al 2015], [Humphries and Prescott 2010], but not cues [Domingues et al 2025]. S.sh is important for place habituation: avoiding places already visited [Floresco 2015]. S.core is more associated with seek actions, and S.msh associated with place preference and avoidance [Fisher et al 2025]. S.ot is less studied, but is strongly Ob and O.pir related. Assigning S.lot to the same group as S.lsh and S.mot with S.msh is not well supported functionally, but does have some transcriptional support [Chen R et al 2021]. S.msh.d generally supports RTPP (real-time place preference) and S.msh.v RTPA (real-time place avoidance) [Ding YD et al 2022], [D’Aquila 2024], [Faget et al 2024], but because other studies report S.msh.d as necessary for cued avoidance [Ramirez et al 2015], the S.msh function may not be as simple as a clear RTPA/RTPP difference. Sv has three-dimensional aspects as well, with S.msh.a (anterior S.msh) and S.msh.p (posterior S.msh) having opposing seek and avoid motivation [Castro et al 2015], [Berridge 2019], [Bond et al 2020], [Marinescu and Labouesse 2024], with differing projections to locomotion vs eating regions [Richard and Berridge 2011].

As a complication for reading the research, because the heterogeneity of Sv regions was relatively recently discovered, many papers report results for S.sh without distinguishing between S.lsh, S.msh.d, or S.msh.v, despite these regions having different or even opposing functions. Older papers often simply report results for Sv without even distinguishing S.core from S.sh. Another complication is that Sv is also important for eating, not simply dedicated to seek or avoidance. Stimulating S.sh.a immediately stops eating [Reed et al 2018]. However, eating and roaming/seeking are related because they’re mutually exclusive: the animal needs ot stop roaming or seeking to eat. In case cases eating circuits may actually be stop-moving circuits, since 70% of S.sh are inhibited while eating and 30% are short lived excite while eating [Marinescu and Labouesse 2024]. Note that although the divisions in S.v are broadly topographic, some sub-functions could be mixed salt-and-pepper style, particularly in the complicated S.msh region.

So, the essay can add S.msh place habituation to avoid places already visited [Floresco 2015]. This seems likely to be S.msh.d because S.msh.v is more associated with threat avoidance [Ding YD et al 2022].

Odor neighborhoods for roaming

The odor spatial hypothesis suggests that the vertebrate Ob (olfactory bulb) is used more for spatial navigation than odor identification [Jacobs 2012]. Odors form spatial neighborhoods [Jacobs 2022], and the mammalian E.hc (hippocampus) place fields are driven by odor [Jacobs 2022]. Odors are rarely in isolation, but are dynamic in space and time. Place cells in blind rats are similar to sighted rats [Marin et al 2021]. A very old model of E.hc called it the rhinencephalon (nose brain), which was displaced by the discovery of spatial place fields, but if place is grounded by an old odor neighborhood circuit, then rhinencephalon may be accurate [Jacobs 2022].

For the essay simulation I’ve created two parallel basal ganglia paths for seek and roam. The seek path is only activated when the animal is following a food odor plume. The roam path is more broadly activated when the animal is searching for food. For specific paths, S.core and S.lsh appear specific to seek [Dobrovitsky 2017], [Soares-Cunha et al 2020], [Walle et al 2024]. Sv research doesn’t investigate roam circuits per se, but RTPA (real-time place avoidance) and RTPP (real-time place preference) and conditioned place preference are centered on S.msh [Britt et al 2012], [Marinescu and Labouesse 2024].

Flowchart illustrating the pathways in the basal ganglia for seeking and roaming behaviors in response to olfactory signals. Left side shows 'Ob.l place' input leading to seek and roam pathways, including interactions between various neural regions. — Paths used by the simulation for seek timeout and roam timeout with odor neighborhood context. H.l (lateral hypothalamus), Hb.lm (lateral habenula, medial part), MLR (midbrain locomotor region), Ob.l (lateral olfactory bulb), Pv.dl (ventral pallidum, dorsolateral part), Pv.vm (ventral pallidum, ventromedial part), R1.a (anterior hindbrain locomotor region), R5.rs (mid-hindbrain turning region), S.core (ventral striatum core), S.msh.d (striatum medial shell, dorsal part), T.pf (parafascicular thalamus), V.rn (raphe nuclei)

The above diagram shows a hypothetical dual seek and roam circuit. S.core uses seek action feedback / efference copy to enable a seek timeout to avoid perseveration. S.msh.d uses H.l (lateral hypothalamus) roaming driver to enable place habituation to avoid searching places already visited.

A screenshot of a simulation showing an animal's movement in a patterned arena with various sections representing different odor neighborhoods. Circles denote active olfactory stimuli and a star marks the location of food. Syllables and phonemes are used to represent odor features. — *Screenshot of the simulation with the animal leaving an area it’s already explored. The phonemes “d-“, “-o-“, and “-g” represent the current odor neighborhood.*

The above screenshot shows the simulation for roaming odor neighborhood. Each pattern represents a different odor neighborhood. The phonemes on the right — “d-“, “-o-“, and “-g” — represent odor features of the neighborhood. The animal is avoiding the bottom neighborhood marked by blue horizontal lines because roaming has timed out for that neighborhood.

Box plot and violin plot comparing the distances traveled in an open field for two groups: with 'PvRoam' and without 'PvRoam'. — *Monte Carlo simulation of the animal’s search. PvRoam represents roaming with timeout enabled. No PvRoam representing roaming without timeout.*

As a test to verify that avoiding place repetition improves food search, I ran 300 Monte Carlo simulations for both the roam timeout enabled and disabled. In the simulation code, the timeout circuit is organized by its Pv projection, which owns the corresponding Sv. So enabling the roaming timeout means enabling PvRoam. The results suggest that avoiding place repetition improves performance by regarding the long-search tail of the distribution.

Discussion: multimodal feature inputs

The essay’s simulation only used odor features as striatum inputs for identifying neighborhoods, but the mosaic model can work with multimodal inputs. For example the lateral line sense can detect a barrier to the right of the animal, That signal can be added to the striatum mosaic to distinguish odor neighborhoods bordered by a reef from a neighborhood over sand. Temperature sensors can distinguish cold and warm neighborhoods. Similarly, even simple, non-imaging photoreceptors tuned to multiple colors could help distinguish sand from reef or deep blue ocean from shallow waters. Three or four bits of visual information could help distinguish neighborhoods without needing complicated visual processing.

Similarly, although the fruit fly mushroom body mainly has odor inputs, it also includes some visual, gustatory, and thermosensory input [Chan ICW et al 2024]. Like a striatum mosaic, the visual processing in the mushroom body isn’t complex, but it can distinguish environments.

Insect mushroom body as a cerebellum-like structure

An interesting comparison between the insect MB (mushroom body) and vertebrate CB-like (cerebellum-like) structures suggests that both act as adaptive sensory filters [Farris 2011]. Vertebrate CB-like structures, mainly in the hindbrain, use anti-Hebbian plasticity to predict and erase self-motion from sensor data [Bell et al 2008], [Montgomery et al 2012].

For example, the aquatic lateral-in sense uses water motion sensors to detect objects and prey. If the animal is swimming near an obstacle to the right, the relative water motion produces a curl around the animal, which a relatively simple circuit can decode to infer the barrier [Oteiza et al 2017]. Because the animal’s own swimming also produces water movement, a CB-like organ R.mon (medial octavo lateral nucleus) subtracts the self signal, enabling more accurate obstacle and prey detection.

Similar to the striatum context and action architecture explored in this essay, CB-like structures have a context formed by parallel fibers, which encodes multimodal combination of self-action and proprioceptive input, and a primary sensory input, which the context modulates. Also similar to this essay’s striatum model, repeated activation is anti-Hebbian: suppressing repeated activation.

There are major differences between the striatum and CB-like functionality, of course. CB-like structures form an adaptive filter to produce a more useful signal, while the striatum timeout in this essay avoid repeating search areas, and it’s hard to find any commonality in those two functions other than the very general avoidance of repetition.

Possible cortex enhancements

Because the simulation is a tow model, it hides the noise and signal problems. Odors in particular are difficult and messy sensor input because odors in water are clumpy, not the clean odor gradients and neighborhoods of the model. Real odor signals will appear and disappear, and neuron signals are generally short, between 3ms for fast AMPA receptors and 100ms NMDA receptors, but the odor needs much longer sustain, on the order of several seconds.

Cortical pyramidal neurons can activate a sustained ADP (afterdepolarization) model lasting on the order of 6-8 seconds when activated by ACh (acetylcholine) activating mACh.q (Gq coupled acetylcholine receptor). This sustained activity could stretch an odor neighborhood signal across the gaps in spotty odor receptor signal. A proto-vertebrate improvement could use a simple proton-cortical circuit as short term memory.

A more complicated improvement is associative pattern recognition. The mosaic striatum model can detect simple patterns, but it can’t generalize, and may be susceptible to noise and distractor signals. Typically, an odor scene will have multiple odor molecules, unlike the simulation’s simplified model. Cortex circuits could filter the noisy inputs and produce a more reliable input to the striatum, replacing direct Ob input with more conceptual O.pir (piriform cortex) engrams.

Odor and place

Although odors can form neighborhoods, they aren’t necessarily precise or reliable. One complicated improvement is dedicated cortical regions devoted to identifying place. O.pir ish the main olfactory cortex in mammals with homologous olfactory cortexes in other vertebrates. In mammals, O.pir.a (anterior O.pir) idenfieis odors, and O.pir.p (posterior O.pir) detects place [Poo C et al 2022]. The neuronal connectivity of O.pir.p resembles parts of E.hc, which is well-known to encode place.

Consider an evolutionary sequence of improvements that starts from a simple striatum mosaic that detects odor neighborhoods, then improves that primitive place detection with more sophisticated cortical place in O.pir.p. That single-mode odor place detection could then combine with other sensory modes using egocentric and allocentric inputs like head direction and landmarks into a more reliable place detection in E.hc.

References

Al-Hasani R, McCall JG, Shin G, Gomez AM, Schmitz GP, Bernardi JM, Pyo CO, Park SI, Marcinkiewcz CM, Crowley NA, Krashes MJ, Lowell BB, Kash TL, Rogers JA, Bruchas MR. Distinct Subpopulations of Nucleus Accumbens Dynorphin Neurons Drive Aversion and Reward. Neuron. 2015 Sep 2;87(5):1063-77.

Beauséjour PA, Auclair F, Daghfous G, Ngovandan C, Veilleux D, Zielinski B, Dubuc R. Dopaminergic modulation of olfactory-evoked motor output in sea lampreys (Petromyzon marinus L.). J Comp Neurol. 2020 Jan 1;528(1):114-134.

Beauséjour, P.A., Zielinski, B. and Dubuc, R., 2022. Olfactory-induced locomotion in lampreys. Cell and tissue research, pp.1-15.

Beauséjour PA, Veilleux JC, Condamine S, Zielinski BS, Dubuc R. Olfactory Projections to Locomotor Control Centers in the Sea Lamprey. Int J Mol Sci. 2024 Aug 29;25(17):9370.

Bell, Curtis C., Victor Han, and Nathaniel B. Sawtell. Cerebellum-like structures and their implications for cerebellar function. Annu. Rev. Neurosci. 31 (2008): 1-24.

Berridge KC. Affective valence in the brain: modules or modes? Nat Rev Neurosci. 2019 Apr;20(4):225-234.

Bond CW, Trinko R, Foscue E, Furman K, Groman SM, Taylor JR, DiLeone RJ. Medial Nucleus Accumbens Projections to the Ventral Tegmental Area Control Food Consumption. J Neurosci. 2020 Jun 10;40(24):4727-4738.

Britt JP, Benaliouad F, McDevitt RA, Stuber GD, Wise RA, Bonci A. Synaptic and behavioral profile of multiple glutamatergic inputs to the nucleus accumbens. Neuron. 2012 Nov 21;76(4):790-803.

Castro DC, Cole SL, Berridge KC. Lateral hypothalamus, nucleus accumbens, and ventral pallidum roles in eating and hunger: interactions between homeostatic and reward circuitry. Front Syst Neurosci. 2015 Jun 15;9:90.

Chan, I.C.W., Chen, N., Hernandez, J., Meltzer, H., Park, A. and Stahl, A., 2024. Future avenues in Drosophila mushroom body research. Learning & Memory, 31(5), p.a053863.

Chen, G., Lai, S., Bao, G., Ke, J., Meng, X., Lu, S., Wu, X., Xu, H., Wu, F., Xu, Y. and Xu, F., 2023. Distinct reward processing by subregions of the nucleus accumbens. Cell reports, 42(2).

Chen R, Blosser TR, Djekidel MN, Hao J, Bhattacherjee A, Chen W, Tuesta LM, Zhuang X, Zhang Y. Decoding molecular and cellular heterogeneity of mouse nucleus accumbens. Nat Neurosci. 2021 Dec;24(12):1757-1771.

Da Cunha C, Wietzikoski EC, Dombrowski P, Bortolanza M, Santos LM, Boschen SL, Miyoshi E. Learning processing in the basal ganglia: a mosaic of broken mirrors. Behav Brain Res. 2009 Apr 12;199(1):157-70.

Dale N. Delayed production of adenosine underlies temporal modulation of swimming in frog embryo. J Physiol. 1998 Aug 15;511 ( Pt 1)(Pt 1):265-72.

D’Ascenzo M, Fellin T, Terunuma M, Revilla-Sanchez R, Meaney DF, Auberson YP, Moss SJ, Haydon PG. mGluR5 stimulates gliotransmission in the nucleus accumbens. Proc Natl Acad Sci U S A. 2007 Feb 6;104(6):1995-2000.

D’Aquila, PS, 2024. Licking microstructure in response to novel rewards, reward devaluation and dopamine antagonists: possible role of D1 and D2 medium spiny neurons in the nucleus accumbens. Neuroscience & Biobehavioral Reviews, p.105861.

Derjean D, Moussaddy A, Atallah E, St-Pierre M, Auclair F, Chang S, Ren X, Zielinski B, Dubuc R. A novel neural substrate for the transformation of olfactory inputs into motor output. PLoS Biol. 2010 Dec 21;8(12):e1000567.

Ding YD, Chen X, Chen ZB, Li L, Li XY, Castellanos FX, Bai TJ, Bo QJ, Cao J, Chang ZK, Chen GM, Chen NX, Chen W, Cheng C, Cheng YQ, Cui XL, Duan J, Fang YR, Gong QY, Hou ZH, Hu L, Kuang L, Li F, Li HX, Li KM, Li T, Liu YS, Liu ZN, Long YC, Lu B, Luo QH, Meng HQ, Peng DH, Qiu HT, Qiu J, Shen YD, Shi YS, Si TM, Tang YQ, Wang CY, Wang F, Wang K, Wang L, Wang X, Wang Y, Wang YW, Wu XP, Wu XR, Xie CM, Xie GR, Xie HY, Xie P, Xu XF, Yang H, Yang J, Yao JS, Yao SQ, Yin YY, Yuan YG, Zang YF, Zhang AX, Zhang H, Zhang KR, Zhang L, Zhang ZJ, Zhao JP, Zhou RB, Zhou YT, Zhu JJ, Zhu ZC, Zou CJ, Zuo XN, Yan CG, Guo WB. Reduced nucleus accumbens functional connectivity in reward network and default mode network in patients with recurrent major depressive disorder. Transl Psychiatry. 2022 Jun 6;12(1):236.

Dobrovitsky V, West MO, Horvitz JC. The role of the nucleus accumbens in learned approach behavior diminishes with training. Eur J Neurosci. 2019 Nov;50(9):3403-3415.

Domingues, A.V., Carvalho, T.T., Martins, G.J., Correia, R., Coimbra, B., Bastos-Gonçalves, R., Wezik, M., Gaspar, R., Pinto, L., Sousa, N. and Costa, R.M., 2025. Dynamic representation of appetitive and aversive stimuli in nucleus accumbens shell D1-and D2-medium spiny neurons. Nature communications, 16(1), p.59.

Faget L, Oriol L, Lee WC, Zell V, Sargent C, Flores A, Hollon NG, Ramanathan D, Hnasko TS. Ventral pallidum GABA and glutamate neurons drive approach and avoidance through distinct modulation of VTA cell types. Nat Commun. 2024 May 18;15(1):4233.

Fang, L.Z. and Creed, M.C., 2024. Updating the striatal–pallidal wiring diagram. Nature neuroscience, 27(1), pp.15-27.

Farris, S.M., 2011. Are mushroom bodies cerebellum-like structures?. Arthropod structure & development, 40(4), pp.368-379.

Fee MS. Oculomotor learning revisited: a model of reinforcement learning in the basal ganglia incorporating an efference copy of motor actions. Front Neural Circuits. 2012 Jun 27;6:38.

Fee MS. The role of efference copy in striatal learning. Curr Opin Neurobiol. 2014 Apr;25:194-200.

Fisher, A.A., Gonzalez, L.S., Cappel, Z.R., Grover, K.E., Waclaw, R.R. and Robinson, J.E., 2025. Dopaminergic encoding of future defensive actions in the mouse nucleus accumbens. PNAS nexus, 4(5), p.pgaf128.

Floresco SB. The nucleus accumbens: an interface between cognition, emotion, and action. Annu Rev Psychol. 2015 Jan 3;66:25-52.

Hook, V., Toneff, T., Baylon, S. and Sei, C., 2008. Differential activation of enkephalin, galanin, somatostatin, NPY, and VIP neuropeptide production by stimulators of protein kinases A and C in neuroendocrine chromaffin cells. Neuropeptides, 42(5-6), pp.503-511.

Humphries MD, Prescott TJ. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog Neurobiol. 2010 Apr;90(4):385-417.

Jacobs L. F. (2012). From chemotaxis to the cognitive map: the function of olfaction. Proc. Natl. Acad. Sci. U.S.A. 109(Suppl. 1) 10693–10700

Jacobs LF. How the evolution of air breathing shaped hippocampal function. Philos Trans R Soc Lond B Biol Sci. 2022 Feb 14;377(1844):20200532.

Kang S, Hong SI, Lee J, Peyton L, Baker M, Choi S, Kim H, Chang SY, Choi DS. Activation of Astrocytes in the Dorsomedial Striatum Facilitates Transition From Habitual to Goal-Directed Reward-Seeking Behavior. Biol Psychiatry. 2020 Nov 15;88(10):797-808.

Konradi, C., Macı́as, W., Dudman, J.T. and Carlson, R.R., 2003. Striatal proenkephalin gene induction: coordinated regulation by cyclic AMP and calcium pathways. Molecular brain research, 115(2), pp.157-161.

Kornfeld, J., Januszewski, M., Schubert, P., Jain, V., Denk, W. and Fee, M.S., 2020. An anatomical substrate of credit assignment in reinforcement learning. BioRxiv, pp.2020-02.

Lahiri AK, Bevan MD. Dopaminergic Transmission Rapidly and Persistently Enhances Excitability of D1 Receptor-Expressing Striatal Projection Neurons. Neuron. 2020 Apr 22;106(2):277-290.e6.

Ma L, Day-Cooney J, Benavides OJ, Muniak MA, Qin M, Ding JB, Mao T, Zhong H. Locomotion activates PKA through dopamine and adenosine in striatal neurons. Nature. 2022 Nov;611(7937):762-768.

Marin AC, Schaefer AT, Ackels T. Spatial information from the odour environment in mammalian olfaction. Cell Tissue Res. 2021 Jan;383(1):473-483.

Marinescu AM, Labouesse MA. The nucleus accumbens shell: a neural hub at the interface of homeostatic and hedonic feeding. Front Neurosci. 2024 Jul 30;18:1437210.

Montgomery, John C., David Bodznick, and Kara E. Yopak. The cerebellum and cerebellum-like structures of cartilaginous fishes. Brain Behavior and Evolution 80.2 (2012): 152-165.

O’Donnell P, Grace AA. Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J Neurosci. 1995 May;15(5 Pt 1):3622-39.

Poo C, Agarwal G, Bonacchi N, Mainen ZF. Spatial maps in piriform cortex during olfactory navigation. Nature. 2022 Jan;601(7894):595-599.

Ramirez F, Moscarello JM, LeDoux JE, Sears RM. Active avoidance requires a serial basal amygdala to nucleus accumbens shell circuit. J Neurosci. 2015 Feb 25;35(8):3470-7.

Reed SJ, Lafferty CK, Mendoza JA, Yang AK, Davidson TJ, Grosenick L, Deisseroth K, Britt JP. Coordinated Reductions in Excitatory Input to the Nucleus Accumbens Underlie Food Consumption. Neuron. 2018 Sep 19;99(6):1260-1273.e4.

Sesack SR, Grace AA. Cortico-Basal Ganglia reward network: microcircuitry. Neuropsychopharmacology. 2010 Jan;35(1):27-47.

Soares-Cunha C, de Vasconcelos NAP, Coimbra B, Domingues AV, Silva JM, Loureiro-Campos E, Gaspar R, Sotiropoulos I, Sousa N, Rodrigues AJ. Nucleus accumbens medium spiny neurons subtypes signal both reward and aversion. Mol Psychiatry. 2020 Dec;25(12):3241-3255.

Suryanarayana, S. M., Perez-Fernandez, J., Robertson, B., & Grillner, S. (2021). Olfaction in lamprey pallium revisited—dual projections of mitral and tufted cells. Cell Reports, 34(1).

Walle R, Petitbon A, Fois GR, Varin C, Montalban E, Hardt L, Contini A, Angelo MF, Potier M, Ortole R, Oummadi A, De Smedt-Peyrusse V, Adan RA, Giros B, Chaouloff F, Ferreira G, de Kerchove d’Exaerde A, Ducrocq F, Georges F, Trifilieff P. Nucleus accumbens D1- and D2-expressing neurons control the balance between feeding and activity-mediated energy expenditure. Nat Commun. 2024 Mar 21;15(1):2543.

Weiss, L., 2020. Information processing in the olfactory system of different amphibian species (Doctoral dissertation, Dissertation, Göttingen, Georg-August Universität, 2020).

Wilson, R.I. and Mainen, Z.F., 2006. Early events in olfactory processing. Annu. Rev. Neurosci., 29(1), pp.163-201.

Yu X, Taylor AMW, Nagai J, Golshani P, Evans CJ, Coppola G, Khakh BS. Reducing Astrocyte Calcium Signaling In Vivo Alters Striatal Microcircuits and Causes Repetitive Behavior. Neuron. 2018 Sep 19;99(6):1170-1187.e9.

16: Classical conditioning and negative learning

On September 4, 2023February 6, 2026 By Scott FergusonLeave a comment

While I was researching how the fruit fly might learn to ignore initially-attractive odors, I ran into a difficulty that most papers aren’t interested in that attractive-to-ignore transition. For this simulation this lack of information means I have needed to guess as to the plausibility. One possible reason for the lack of data might be an over-focus on specifics of classical associative learning.

Foraging and odor-following task

The task here is finding food by following a promising odor. As explored in the essay 15 and essay 16 simulations, this following-odor task is more complicated than it first appears because a naive solution leads to perseveration: never giving up on the odor. Perseveration is potentially fatal for the animal if it can’t break away from a non-rewarding lead, an inability to accept failure. To avoid this fatal flaw, there needs to be a specific circuit to handle that failure, otherwise the animal will follow the odor forever. At the same time, the animal must spend some time exploring the potential food area before giving up (patience). This dilemma is similar to the explore/exploit dilemma for foraging theory and reinforcement learning. A state diagram for the odor-foraging task might look something like the following:

*Behavior state transitions for following an odor to food.*

This odor task starts when the animal detects the odor. The animal approaches the odor repeatedly until it either finds food, the odor disappears, or the animal gives up. Giving up is the interesting case here because it requires an internal state transition, while all other inputs causes stimulus-reponse transitions: the animal just reacts. Finding food triggers consummation (eating), and disappearing odor voids the task. Both are external stimuli. In contract, giving up requires internal state circuitry to decide when to quit, a potentially difficult decision.

Learning to ignore

The animal can improve its odor-following performance if it can learn to ignore false leads, as explored in essay 15. The animal follows intrinsically-attractive odors and if there’s a reward it continues to approach the odor. But if the animal doesn’t find food, it remembers the odor and ignores the odor the next time. The simulations in essay 15 showed the effectiveness of this strategy, improving food discovery.

*Learning transitions for an intrinsically attractive odor.*

In the learning diagram above, when the animal finds food in an odor that predicts food, it maintains the intrinsic approach. When the animal doesn’t find food for that odor, it will ignore the odor for the next time. This learning is not simple habituation because the learning depends on the reward outcome, but it’s also not classical associative learning.

Classical associative learning

Classical associative learning (Pavlovian) starts from an initial blank state (tabula rasa) and learns from reward to approach and from punishment to avoid. The null transition, no reward and no punishment, maintains the initial blank state, although this assumption is implicit and not discussed as an important part of the model. A number of points about the classical associative learning model:

First, the greyed-out transition is an assumption, often untested or if it is tested, it’s dismissed as unimportant. For example, [Tempel et al. 1983] notes that OCT (a testing odor) becomes increasingly aversive even without punishment (a shock), which contradicts the greyed transition, but doesn’t incorporate that observation into the analysis. Similarly, [Berry et al. 2018] found that the test odors have increasing LTD (long-term depression) even without punishment or reward, but relegates that observation as unpublished data, presumably because it was irrelevant to the classical model in the study.

Second, classical association is a blank slate (tabula rasa) model: the initial state is an ignore state. Although fruit flies have intrinsically attractive odors and intrinsically repelling odors, research seems to focus on neutral odors. possibly because classical association expects an initial ignore state. But starting with neutral odors means there’s little data about learning with intrinsically attractive odors. For example attractive odors might be impervious to negative learning. In fruit flies, the lateral horn responds to non-learned behavior, such as intrinsic attractive odors. The mushroom body (learning) might not suppress the lateral horn’s intrinsic behavior for those odors.

*Learning transitions for a semi-classical situation where the non-reward transition learns aversion.*

Third, in the fruit fly it’s unclear whether a no-reward transition uses the same MBONs as a punishment transition when the only negative learning data is from punishment studies. For example, both γ1-MBON and γ2-MBON are short term memory aversive-learning MBONs, as well as possibly α’3-MBON. A study that does test the no-reward transition as well as the punishment transition can distinguish between the two. [Hancock et al. 2022] includes a non-reward test to narrow the punishment effect to γ1, and notes non-reward depression for γ2, γ3, and γ4, but still treats and non-associative depression as outside the scope of interest.

Alternative learning model

An alternative is to treat all learning transitions as equally important, as opposed classical association’s focus on one a few transitions.

*Alternative learning where all transitions are treated as interesting.*

In the above diagram, seven of the nine transitions are interesting, and even the two trivial transitions, rewarded intrinsic-approach and punished intrinsic repel, are interesting in context of other transitions, because the implementing circuits might need to remember the reward.

Foraging learning revisited

Returning to the original foraging learning problem:

Learning for an instrinsically-attractive odor. — *Learning transitions for an intrinsically-attractive odor.*

How might this transition be implemented in the fruit fly mushroom body? A single-neuron implementation might be possible if a reward can reverse a habituation-like LTD for the none transition, such as a dopamine spike leading to LTP (long-term potentiation). A dual-neuron implementation might use one neuron to store the reward vs non-reward state and the other to store a visited-recently data, such as the surprise neuron α’3 [Hattori et al. 2017].

References

Berry JA, Phan A, Davis RL. Dopamine Neurons Mediate Learning and Forgetting through Bidirectional Modulation of a Memory Trace. Cell Rep. 2018 Oct 16

Hancock, C.E., Rostami, V., Rachad, E.Y. et al. Visualization of learning-induced synaptic plasticity in output neurons of the Drosophila mushroom body γ-lobe. Sci Rep 12, 10421 (2022).

Hattori, Daisuke, et al. Representations of novelty and familiarity in a mushroom body compartment. Cell 169.5 (2017): 956-969.

Tempel, Bruce L., et al. Reward learning in normal and mutant Drosophila. Proceedings of the National Academy of Sciences 80.5 (1983): 1482-1486.

Essay 16: Learning to Ignore

On August 22, 2023February 6, 2026 By Scott FergusonLeave a comment

Essay 16 extends the odor seeking (chemotaxis) of essay 15 by adding a single memory item. The memory caches a failed odor search, avoiding the cost of searching for false odors. The neuroscience source is the fruit fly Drosophila. The simulation is still based on a Braitenberg slug with distinct circuits for chemotaxis and for obstacle avoidance.

Mushroom body model

The fruit fly mushroom body (MB) is the learning center. MB is a modulating system: if it’s knocked out, the fruit fly behaves normally, although with only intrinsic, unlearned behavior. Essay 16 focuses on a single MBON short term memory (STM) output neuron, which may specifically be the γ2 neuron.

Architecture of fruit fly mushroom body. — *Mushroom body architecture, adapted from [Aso et al. 2014]*

For simplicity and focus, essay 16 isn’t implementing the KC yet. Instead, the γ2 MBON receives its input directly from a small number of odor projection neurons (PN) that was implemented in essay 15. Essentially, the input is a small set of primitive odors, where the full KC is a massive combinatorial odor spectrum.

Candidate odors from evolution

To motivate why evolution might develop learning, consider the food-seeking slug from essay 15. Since the food odor in essay 15 perfectly predicted food, there was no reason to learn anything about food. The simulation’s “evolution” has perfectly solved the artificially perfect world, selecting exactly those odors needed to find food.

Choosing the right set of candidate odors is a dilemma for evolution. Too many candidates means wasted search time. Too few candidates avoids wasted time, but misses out on opportunities, which may be a smaller problem than too many candidates because the animal can fall back to random, brownian-motion search. This beast against including semi-predictive odors might mean that early Precambrian evolution might only favor the highest predictors and skip semi-productive odors.

*Candidate odor surrounded by distractors*

The preceding image represents odors potentially available to the slug from an evolutionary design perspective. If the beige color is the only candidate for food, the slug will ignore the blue-ish colors because it never senses the odor. There’s no need for a circuit or behavior to distinguish the two. For the animal, those odors don’t exist.

Food odors don’t perfectly predict food, either because of lingering odors or simply candidate odors that aren’t always from nutritious food. For example, the fruit fly can taste sweet and it can also sense nutrition from a rest in blood sugar. That distinction between sweet and nutritious is reflected in the mushroom body with specific neurons for each [Owald et al. 2015].

Classical association

For this essay, let’s explore what could be the most trivial memory, in the context of the fruit fly MB. The MB output has only 24 neurons in 15 distinct compartments per hemisphere. Each compartment appears to have specialized roles, such as short term memory (STM) vs long term memory (LTM) [Bouzaiane et al. 2015], and water seeking compared to sugar seeking [Owald et al. 2015].

Although learning studies typically use classical association (Pavlovian) terminology, where a conditioned stimulus (CS) like the food odor becomes associated with an unconditioned stimulus (US) like consuming food, I don’t think that framing is useful for the odor-seeking behavior of the simulated slug.

*Naive animal missing food before classical training*

In the diagram above, which follows the classical model, the animal (arrow) missing the food (brown square) despite being in the candidate odor’s plume because it hasn’t learned the associate the odor (CS) with the food (US). It only learns the association if it finds the food through brownian random search. Even then, if it randomly hits another food source with a different odor, it will forget the first, limiting the gain from this learning.

Even the non-learning algorithm of essay 14 performs better, because naive searching of all candidate odors is relatively successful, even if slightly time inefficient. Behaviorally, the difference is between default-approach or default-ignore. Default-approach needs to learn to ignore and default-ignore needs to learn to approach.

Learning to ignore

Learning to ignore is an alternative to the classical associative way of looking at the problem. It’s not an argument against classical conditioning in total, but it is a different perspective that highlights different features of the problem.

*Successful approach to food and failure*

The diagram above shows a successful approach to food and an unsuccessful approach. Both candidate odors potentially signal food because evolution ignores useless odors, but in this neighborhood the reddish odor is a non-food signal. As in essay 14, habituation will rescue the animal from perseveration, spending infinite time exploring a useless odor, but once an odor is found useless, ignoring it from the start would improve search efficiency.

Since odors in a neighborhood are likely similar, the animal is likely to encounter the useless odor soon. So, remembering a single item, like a single item cache, will improve the search by avoiding cost, until the animal reaches an area that does have nutritious food. The single-item cache lets the animal ignore patches of non-predictive odors.

Single item cache (short term memory)

A single mushroom body output neuron (MBON) and its associated dopamine neuron (DAN) can implement a single item cache by changing the weights of the KC to MBON synapses with long-term depression (LTD). Following the previous discussion, since it’s more efficient to remember the last failure than the last success, the learning is LTD at the synapse between the odor and the MBON. In fruit flies, short term memory (STM) is on the order of 2h. (For a fuller discussion between “short” and “long” term see [Sossin 2008].)

*Reduced MB circuit for negative learning*

In the above diagram, the O2 synapse with a ball represents the LTD cache item. If the animal senses odors for either O1 or O3, it approaches the odor. If it senses O2, it ignores the odor because of the LTD at the synapse. (The colors follow the mnemonic model. Purple represents primary sensor/odor, and blue represents apical/limbic/odor and motivation areas.)

The DAN needs to implement a failure signal to implement LTD, which is actually relatively complicated. Unlike success, which has an obvious direct stimulus when finding food, failure is ambiguous. How long the animal should persist before giving up is a difficult problem, and at very least requires a timer even for the simplest strategy. Because habituation already implements a timeout, an easy solution is to copy the circuit or possibly use its output. So, if the animal exists the odor plume because of habituation, the DAN might signal failure.

Another possibly strategy is for the DAN to continuously degrade the active signal as in habituation, and only rescue the synapse when discovering food. Results from [Berry et al. 2018] show the needed degrading over time (ramping LTD) in their study of MBON-γ2, although that study didn’t explore the rescuing of approach by finding food that we seed.

So, the second strategy might require a second, opposing neuron, which I’ll probably explore later. For this essay, the DAN will produce a failure signal on timeout and a success signal on finding food, something like a reward prediction error signal from reinforcement learning [Sutton and Barto 2018], but without using a reinforcement learning architecture.

Mammalian correlates

In mammals, the KC/MBON synapse with DAN modulation circuit functionally resembles the hippocampus CA3 to CA1 synapse (E.hc, E.ca1, E.ca3) with locus coerulus (V.lc). In mammals, V.lc is known as the primary source of noradrenaline, associated with surprise and orientation, but it also contains dopamine neurons, and strongly innervates the hippocampus.

[Aston-Jones and Cohen 2005] discuss the locus coeruleus involvement in decision-making, specifically in explore vs exploit decision. If time passes and exploitation continues to fail by not finding food, V.lc signaling encourages moving on and exploring different options, a behavior similar to ours.

The E.ca3 to E.ca1 connection (and E.ec, entorhinal cortex) is believed to detect novelty, and V.lc is active during exploration, Like the fruit fly MBON, the hippocampus uses LTD to learn a new place, using V.lc signal [Lemon et al. 2012] like the DAN.

In contrast with my simulated slug, since the E.hc novelty output doesn’t directly drive food approach, because the mammalian brain is far more complex and abstract, the comparison isn’t exact, but it is an interesting similarity.

Simulation: shared habituation

In essay 15, the simulated slug approached an intrinsically attractive odor to find food, but needed a habituation circuit to avoid perseveration. The fruit fly LN1 neurons between the ~50 main olfactory sensory neurons (ORN) and the ~150 olfactory projection neurons (PN) implement primary olfactory habituation. In this essay, I’m essentially adding a second odor to the system. Although the fruit fly has separate habituation circuits for each of the 50 primary odors, it’s interesting to see what a shared habituation circuit might look like in the simulation.

*Shared habituation pre-learning circuit*

The simulation heat map shows the animal spends much of its time between the odor plumes because the habituation timeout keeps refreshing, despite encountering different odors. While habituation is active, the animal doesn’t approach either odor plume, but mostly moves in the default semi-random pattern. Only when habituation times out will it approach a new odor.

*Shared habituation. Warm colors are food-predicting odors, blue are distractors.*

Split habituation

The fruit fly has a split habituation unlike the previous simulation. Each primary odor has an independent habituation circuit, which is synapse specific.

In the simulation of split habituation, the animal spends more time investigating the odors because each new odor has its own habituation timeout. It can move from a failed odor and immediately explore a new odor.

*Simulation heat map for split habituation*

Although the animal spends much of its time exploring the distractor candidate odors, it’s still a big improvement over random search, because it’s more likely to find food instead of a near miss.

Single distractor

Since a single distractor exactly fits the single item cache, it’s unsurprising that adding the cache immediately solves the distractor problem. In the following heat map, the animal only explores the successful candidate odor and ignores the distractor.

*Simulation heat map for single-item negative cache*

Multiple odors and distractors

Multiple distractor odors is more interesting for a single item cache because it introduces miss-rate as a prominent issue and allows comparison between negative caching and positive caching (classical association). The table below is a summary of feeding time as a success metric for each strategy.

Algorithm	Feeding time
No odor approach	0.8%
No learning	7.4%
LTD (cache)	8.0%
LTP (classical)	5.7%

Success comparison for multiple algorithms, measured by feeding time

No odor approach

As a baseline, the first simulation disables all odor approach. The animal only reaches food when it runs into it randomly. While it’s above the food, the animal will slow, improving its efficiency somewhat. This strategy was explored in essay 14, and resembles the feeding of Trichoplax in [Smith et al. 2015].

*Simulation heat map for odor-ignoring animal.*

As the heat map shows, this strategy is pretty terrible. Because the animal only finds food by randomly crossing it, its success rate is purely a matter of the area covered by food. Although this strategy may have been effective with Precambrian bacteria mats, where finding food isn’t an issue, it’s a problem when finding food is a necessary task.

No learning

Intrinsic chemotaxis is important as a baseline for the learning strategies. In the fruit fly intrinsic odor approach behavior is in the lateral horn. When the MB is disabled, the lateral horn continues to approach odors.

*Simulation heat map for non-learning odor-seeking animal.*

As the above heat map shows, intrinsic odor approach is a vast improvement over non-chemotaxis, improving food time from 0.8% to 7.4% in this environment.

Negative caching (LTD)

The single item caching that’s the focus of this post improves the food time from 7.4% to 8% by avoiding some of the time spent on non-food odors. The difference isn’t as dramatic as adding odor approach itself, but it’s an improvement.

In this strategy, the animal remembers the last failure odor, and ignores the odor plume the next time it reaches it. The animal explores all other odors, including failure odors. On a cache miss (failure), the animal remembers the new failure and forgets the old one.

Classical conditioning (LTP)

The next strategy tries to simulate what classical conditioning might look like if it was used for behavior. In this simulation, the animal only follows the odor after it’s associated with food, which means the animal needs to randomly discover the food first.

*Simulation heat map for single-item classical association learning*

This strategy is actually worse than the non-learning case, because it only finds one food source at a time. Although the heat map shows both being visited, the areas are actually alternating. One source is the only found food for a long time until the other is randomly discovered, when the roles switch and the first is now ignored.

Simulation limitations

I think it’s important to point out some simulation limitations, particularly since I’ve added performance numbers for comparison. The simulation environment and timings can affect the numbers dramatically. For example, the odor plume size dramatically affects the classical conditioning algorithm. If finding food without following the odor is difficult, the classical conditioning animal will have great difficulty finding a new odor.

Specifically, if the gain from following the odor is large, then classical conditioning will always have a penalty, because it loses out on that gain until it makes its association. In contrast an explore-first strategy will always gain the odor-exploring advantage. If the gain of explore-first outweighs its cost, then a non-learning explore-first will win against associative learning.

			
cost_cache = p_miss * cost_miss + (1 - p_miss) * cost_hit
cost = cost_cache + cost_noncache

Consider the rough cache cost model above to see some of the issues with the negative cache. If the non-cacheable cost greatly outweighs the cache miss cost, then it doesn’t matter if the animal learns to avoid irrelevant odors. Contrariwise, if the miss cost is very large, then the miss rate is critical.

In addition, the miss rate is highly dependent on spatial and temporal locality. If similar odors are tightly grouped, even a small cache will have a low miss rate. But if there are many different distractor types spread randomly, the cache will miss most of the time.

References

Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, Dionne H, Abbott LF, Axel R, Tanimoto H, Rubin GM. “The neuronal architecture of the mushroom body provides a logic for associative learning.” Elife. 2014

Aston-Jones, Gary, and Jonathan D. Cohen. “An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.” Annu. Rev. Neurosci. 28 (2005): 403-450.

Berry JA, Phan A, Davis RL. “Dopamine Neurons Mediate Learning and Forgetting through Bidirectional Modulation of a Memory Trace.” Cell Rep. 2018

Bouzaiane E, Trannoy S, Scheunemann L, Plaçais PY, Preat T. “Two independent mushroom body output circuits retrieve the six discrete components of Drosophila aversive memory.” Cell Rep. 2015 May 26;11(8):1280-92.

Lemon N, Denise Manahan-Vaughan, “Dopamine D1/D5 Receptors Contribute to De Novo Hippocampal LTD Mediated by Novel Spatial Exploration or Locus Coeruleus Activity, Cerebral Cortex,” Volume 22, Issue 9, September 2012, Pages 2131–2138.

Owald D, Felsenberg J, Talbot CB, Das G, Perisse E, Huetteroth W, Waddell S. “Activity of defined mushroom body output neurons underlies learned olfactory behavior in Drosophila“. Neuron. 2015 Apr 22

Smith CL, Pivovarova N, Reese TS. “Coordinated Feeding Behavior in Trichoplax, an Animal without Synapses.” PLoS One. 2015 Sep 2

Sossin, Wayne S. “Defining memories by their distinct molecular traces.” Trends in neurosciences 31.4 (2008): 170-175.

Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. MIT press.

15: Seeking Food: Perseveration and Habituation

On August 15, 2023February 5, 2026 By Scott FergusonLeave a comment

Essay 15 is adding food-seeking to the simulated slug. Before the change in essay 14, the slug didn’t seek from a distance, but it does slow when it’s above food to improve feeding efficiency. The slug doesn’t have food-approach behavior, but it does have consummatory behavior. Because the slug doesn’t seek food, it only finds food when it randomly crosses a tile. Most of its movement is random, except for avoiding obstacle.

Essay 15: exploring the fruit fly mushroom body

In the screenshot above, the slug is moving forward with no food senses and no food approach. Its turns are for obstacle avoidance. The food squares have higher visitation because of the slower movement over food. Notice that all areas are visited, although there is a statistical variation because of the obstacle.

Although the world is tiled for simulation simplicity, the slug’s direction, location, and movement is floating-point based. The simulation isn’t an integer world. This continuous model means that timing and turn radius matters.

The turning radius affects behavior in combination with timing, like the movement-persistence circuit in essay 14. The tuning affects the heat map. Some turning choices result in the animal spending more time turning in corners when the dopamine runs out. This turn-radius dependence occurs in animals as well. The larva zebrafish has 13 stereotyped basic movements, and its turns have different stereotyped angles depending on the activity.

Food approach

Food seeking adds complexity to the neural circuit, but it’s still uses a Braitenberg vehicle architecture. (See the discussion on odor tracking for avoided complexity.) Odor sensors stimulate muscles in uncrossed connections to approach the odor. Touch sensors use crossed connections to avoid obstacles. For simultaneous odor and touch, an additional command neuron layer resolves the conflict to favor touch.

The command neurons correspond to vertebrate reticulospinal neurons (B.rs) in the hindbrain. Interestingly, the zebrafish circuit does seem to have direct connections from the touch sensors to B.rs neurons, exactly as pictured. In contrast, the path from odor receptors to B.rs neurons is a longer, more complicated path.

For the slug’s evolutionary parallel, the odor’s attractively is hardcoded, as if evolution has selected an odor that leads to food. Even single-celled animals follow attractive chemicals and avoid repelling chemicals, and in mammals some odors are hardcoded as attractive or repelling. For now, no learning is occurring.

Perseveration (tunnel vision)

Unfortunately, this simple food circuit has an immediate, possibly fatal, problem. Once the slug detects an odor, it’s stuck moving toward it because our circuit can’t break the attraction. Although the slug never stops, it orbits the food scent, always turning toward it. The hoped-for improvement of following an odor to find food is a disaster.

In psychology, this inability to switch away is called perseveration, which is similar to tunnel vision but more pathological. Once a goal is started, the person can’t break away. In reinforcement learning terminology, the inability to switch is like an animal stuck on exploiting and incapable of breaking away to explore.

In the screenshot, the heat map shows the slug stuck on a single food tile. The graph shows the slug turning counter-clockwise, slowed down (sporadically arrested) over the food.

To solve the problem, one option is to re-enabled satiation for the simulation, as was added in essay 14, but satiation only solves the problem if the tile has enough food to satiate the animal. Unfortunately, the tile might have a lingering odor but no food, or possibly an evolutionary food odor that’s unreliable, only signaling food 25% of the time. Instead, we’ll introduce habituation: the slug will become bored of the odor and start ignoring it for a time.

In the fruit fly the timescale for habituation is about 20 minutes to build up and 20-40 minutes to recover. The exact timing is likely adjustable because of the huge variability in biochemical receptors. So, 20 minutes probably isn’t a hard constant across different animals or circuits for habituation, but more of a range between a few minutes or an hour or so.

Because the minutes to hour timescale for habituation is much wider than the 5ms to 2s range for neurotransmitters, the biochemical implementation is very different. Habituation seems to occur by adding receptors to increase receptor and/or adding more neurotransmitter generators to produce a bigger signal.

Fruit fly odor habituation

[Das 2011] studies the biochemical circuit for fruit fly odor habituation. The following diagram tries to capture the essence of the circuit. The main, unhabituated connection is from the olfactory sensory neuron (ORN) to the projection neuron (PN), which projects to the mushroom body. The main fast neurotransmitter for insects is acetylcholine (ACh), represented by beige.

The key player in the circuit is the NMDA receptor on the PN neuron’s dendrite, which works with the inhibitory LN1 GABA neuron to increase inhibition over time to habituate the odor.

The LN1 neuron drives habituation. Its GABA neurotransmitter release inhibits PN, which reduces the olfactory signal. Because LN1 itself can be inhibited, this circuit allows for a quick reversal of habituation. Habituation adds to simple inhibition by increasing the synapse strength (weight) over time when its used and decreasing the weight when its idle.

An NMDA receptor needs both a chemical stimulus (glutamate and glycine) and a voltage stimulus (post-synaptic activation, PN in this case). When activated, it triggers a long biochemical chain with many genetic variations to change synapse weight. In this case, it triggers a retrograde neurotransmitter (such as nitrous oxide, NO) to the pre-synaptic LN1 axon, directing it to add new GABA release vesicles. Because adding new vesicles takes time (20 minutes) and removing the vesicles also takes time (20-40 minutes), habituation adds longer, useful behavior that will help solve the odor perseveration problem.

Slug with habituation

The next slug simulation adds trivial habituation following the example of the fruit fly. The simulation adds a value when the slug senses an odor and decrements the value when the slug doesn’t detect an odor. When habituation crosses a threshold, the odor sensor is cut off from the command neurons. The behavior and heat map look like the following:

As the heat map shows, habituation does solve the fatal problem of perseveration for food approach. The slug visits all the food nodes without having any explicit directive to visit multiple nodes. Adding a simple habituation circuit, creates behavior that looks like an explore vs exploit pattern without the simulation being designed around reinforcement learning. Explore vs exploit emerges naturally from the problem itself.

In the screenshot, the bright tile has no intrinsic meaning. Because habituation increases near any food tile, it can only decay when away from food. That bright tile is near a big gap that lets habituation to drop and recharge.

The code looks like the following:

			
impl Habituation {
  fn update_habituation(&mut self, is_food_sensor: bool) {
    if is_food {
      self.food = (self.food + Self::INC).min(1.);
    } else {
      self.food = (self.food - Self::DEC).max(0.);
    }
  }
  fn is_active(&self) -> bool {
    self.food < Self::THRESHOLD
  }
}

		

Discussion

The essays aren’t designed as solutions; they’re designed as thought experiments. So, their main value is generally the unexpected implementation details or issues that come up in the simulation. For example, how habituation thresholds and timing affect food approach. The bright tile in the last heat map occurs because that food source is isolated from other food sources.

If the slug passes near food sources, the odor will continually recharge habituation and it won’t decay enough to re-enable chemotaxis. The slug needs to be away from food for some time for odor tracking to re-enable. In theory, this behavior could be a problem for an animal. Suppose the odor range is very large and the animal is very slow, like a slug. If simple habituation occurs, the slug might habituate to the odor before it reaches the food, making it give up too soon.

As a possible solution, the LN1 inhibitory neuron that implements habituation could itself be disabled, although that beings back the issue of the animal getting stuck. But perhaps it would instead be diminished instead of being cut off, giving the animal persistence without devolving into perseveration.

Odor vs actual food

Another potential issue is the detection of food itself as opposed to just its odor. If the food tile has food, the animal should have more patience than if the tile is empty with just the food odor. That scenario might explain why other habituation circuits include serotonin or dopamine as modulators. If food is actually present, there should be less habituation.

The issue of precise values raises calibration as an issue, because evolution likely can’t precisely calibrate cutoff values or calibrate one neuron to another. Some of the habituation and related synapse adjustment may simply be calibration, adjusting neuron amplification to the system. In a sense, that calibration would be learning but perhaps atypical learning.

Essay 16: Learning to ignore

References

Das, Sudeshna, et al. “Plasticity of local GABAergic interneurons drives olfactory habituation.” Proceedings of the National Academy of Sciences 108.36 (2011): E646-E654.

Shen Y, Dasgupta S, Navlakha S. Habituation as a neural algorithm for online odor discrimination. Proc Natl Acad Sci U S A. 2020 Jun 2;117(22):12402-12410. doi: 10.1073/pnas.1915252117. Epub 2020 May 19. PMID: 32430320; PMCID: PMC7275754.

Essay 15: Exploring Fruit Fly Mushroom Bodies

On August 7, 2023February 5, 2026 By Scott FergusonLeave a comment

In the fruit fly Drosophila, the mushroom body is an olfactory associative learning hub for insects, similar to the amygdala (or hippocampus or piriform, O.pir) of vertebrates, learning which odors to approach and which to avoid. Although the mushroom body (MB) is primarily focused on olfactory senses, it also receives temperature, visual and some other senses. The genetic markers for MB embryonic development in insects match the markers for cortical pyramidal neurons and other learning neurons like the cerebellum granule cells.

While most studies seem to focus on the mushroom body as an associative learning structure, others like [Farris 2011] compare it to the cerebellum for action timing and adaptive filtering.

Essay 15 will likely expand the essay 14 slug that avoided obstacles, adding odor tracking. The mushroom body should show how to improve the slug behavior without adding too much complexity.

Mushroom body architecture

The mushroom body is a highly structured system, comprised of Kenyon cells (KC), mushroom body output neurons (MBONs), and dopamine (DAN) and optamine (OAN) neurons (collectively MBIN – mushroom body input neurons). The outputs are organized into exactly output compartments, each with only one or two output neurons (MBONs) [Aso et al. 2014].

The 50 primary olfactory neurons (ORN) and 50 projection neurons (PN) project to 2,000 KC neurons. Each KC neuron connects to 6-8 PNs with claw-like dendrites. The 2,000 KC neurons then converge to 34 MBONs. In vertebrates, divergence-convergence pattern occurs in the cerebellum with granule cells to Purkinje cells, in the hippocampus with dentate gyrus to CA3, in the striatum with MSN (medium spiny neurons) to S.nr/S.nc (substantia nigra), and in cortical areas with large granular areas.

Habituation (ORN, PN, LN1)

Olfactory habituation is handled by the LN1 circuit from the olfactory receptor neuron (ORN) to projection neuron (PN) connection. The fly will ignore an odor after 30 minutes, and the odor remains ignored for 30 to 60 minutes. Since the LN1 GABA inhibition neuron implements the habituation, suppressing LN1 can reverse habituation nearly instantly.

Habituation is odor-specific because it’s synapsed based, but a single neuron inhibits multiple odors. (There are multiple LN1 inhibitors, but they’re not odor-specific and inhibit multiple odors.) Habituation is odor-specific because it inhibits on a per-synapse system. LN1 doesn’t inhibit the PN, but inhibits the synapse from an ORN to the target PN. So, only figuring PNs trigger habituation, despite relying on a shared inhibitory neuron. [Shen 2020].

Sparse KC firing (KC, APL)

The 50 projection neurons (PNs) diverge into 2,000 Kenyon cells(KC). Each KC has 6-8 claw-like dendrites to 6-8 PNs. Each fly has a random connection between the PNs and its KCs. The KCs that fire for an odor form a “tag,” which is essentially a hash of the odor inputs.

About 5% of the KCs fire for any odor, which is strictly enforced by the APL circuit. APL is a single inhibitory GABA neuron per side that reciprocally connects with all 2,000 KCs. The APL inhibition threshold ensures only the strongest 5% of KCs will fire.

A computer algorithm by [Dasgupta et al. 2017] uses this KC tag for a “fly hash” algorithm, which computes a hash using a random, sparse connection for locality-sensitive hashing (LSH). LSH retains information from the origin data to retain distance between hashes as reflective as distance between odors. Interestingly, they use the fly hash with visual input, treating pixels as equivalent to odors. Despite the significant differences between unique odors and arbitrary pixels, the results for the fly hash are better than other similar LSH systems.

Dopamine and compartments (MBON, DAN)

The output compartments for the mushroom body are broadly organized into attractive and repellant groups with seven repelling compartments and nine attracting compartments. Three of the compartments (g1-g3) feed forward into other compartments.

Each compartment is fed by one or two dopamine neuron types (DAN), for a total of 20 DAN types for 16 MBON compartments. The repellant section (PPL1) is even more restricted with exactly one dopamine neuron per compartment. The attractive section (PAM) has a bound 20 DAN cells per compartment.

In a study of larval Drosophila [Eichler et al. 2020] studied the entire connectome of the mushroom body and found highly specific reciprocal MBON connections, inhibitory, directly excitatory and axon excitatory.

An [Eschbach et al. 2020] study looked at dopamine (DAN) connective. Each DAN type has distinct inputs from raw value stimuli (unconditioned stimuli – US), and also internal state, and feed back from the mushroom body. So, while the DANs do convey unconditioned stimuli for learning, they’re not a simplistic reward and punishment signal.

Essay 15 simulation direction

Essay 15 will likely continue the slug-like animal from essay 14, and add odor approaching. Because I think the slug will need habituation immediately, I’ll probably explore habituation first.

The essay will probably next explore a single KC to a single MBON, simulating a single odor.

I’m tempted to explore multiple MBON compartments before exploring multiple KCs. Partially for general contrariness: pushing against the simple reward/punishment model to see if it goes anywhere, or if multiple dopamine sources needlessly complicated learning.

And, actually, since I want to push against learning as a solution for everything, to see how much of the mushroom body function can work without any learning at all, treating habituation as short term memory, not learning.

References

Dasgupta S, Stevens CF, Navlakha S. “A neural algorithm for a fundamental computing problem.” Science. 2017 Nov 10;358(6364):793-796. doi: 10.1126/science.aam9868. PMID: 29123069.

Eichler, Katharina, et al. “The complete connectome of a learning and memory centre in an insect brain.” Nature 548.7666 (2017): 175-182.

Eschbach, Claire, et al. “Recurrent architecture for adaptive regulation of learning in the insect brain.” Nature Neuroscience 23.4 (2020): 544-555.

Farris, Sarah M. “Are mushroom bodies cerebellum-like structures?.” Arthropod structure & development 40.4 (2011): 368-379.

Shen Y, Dasgupta S, Navlakha S. “Habituation as a neural algorithm for online odor discrimination.” Proc Natl Acad Sci U S A. 2020 Jun 2;117(22):12402-12410. doi: 10.1073/pnas.1915252117. Epub 2020 May 19. PMID: 32430320; PMCID: PMC7275754.

Striatum as timeout

Action and context as striatum inputs

Innate and contextual odors

Odor neighborhoods

Seek striatum

Fruit fly mushroom body and Kenyon cells

S.core seek and S.msh roam

Odor neighborhoods for roaming

Discussion: multimodal feature inputs

Insect mushroom body as a cerebellum-like structure

Possible cortex enhancements

Odor and place

References

Foraging and odor-following task

Learning to ignore

Classical associative learning

Alternative learning model

Foraging learning revisited

References

Mushroom body model

Candidate odors from evolution

Classical association

Learning to ignore

Single item cache (short term memory)

Mammalian correlates

Simulation: shared habituation

Split habituation

Single distractor

Multiple odors and distractors

No odor approach

No learning

Negative caching (LTD)

Classical conditioning (LTP)

Simulation limitations

Links

References

Food approach

Perseveration (tunnel vision)

Fruit fly odor habituation

Slug with habituation

Discussion

Odor vs actual food

References

Mushroom body architecture

Habituation (ORN, PN, LN1)

Sparse KC firing (KC, APL)

Dopamine and compartments (MBON, DAN)

Essay 15 simulation direction

References