The simple phototaxis implementation exposes a few problem with the simulation, both from running it and from reviewing neuroscience to critique it.
Interrupts
The essay doesn’t currently implement any interrupt mechanism. When running into darkness, the animal turns around and starts an area restricted search (ARS), but if the animal is in the middle of a long Levy path, it will cross the border and the search will not find the border.
Long initial paths breaks the phototaxis algorithm.
The problem here is that the ARS starts too late because the animal doesn’t interrupt the current behavior when encountering the border.
One solution is to create an interrupt (orientation) system, which exists in the vertebrate brain in V.ppt (peduncular pontine nucleus), and uses ACh (acetylcholine) to interrupt the current behavior. A natural location for the interrupt is V.ppt for the signal and the stratum as the plan representation, interruptible via ACh interrupts to the striatum.
Another solution is to avoid the uninterruptible behavior entirely, where the problem is the essay’s Levy walk implementation. The essay pre-computes the length of a run instead of continuously creating extensions. In contrast the zebrafish larva swims in bouts, but longer runs are made of multiple forward bouts.
Zebrafish random walk (ARTR area)
The essay’s random walk does not match actual zebrafish search behavior. The essay uses a turn-and-run model where the turn and run length are computed randomly. The zebrafish has a hindbrain oscillator, the ARTR (anterior rhombencephalic turning region) which selects left and right turns [Karpenko et al 2020].
Speculative directed random walk with the zebrafish ARTR. The isthmus is the midbrain-hindbrain boundary (MHB). B.artr anterior rhombencephalic turning region, B.rs reticulospinal motor command, Hb.m medial habenula, M.ip interpeduncular nucleus.
In zebrafish, turns and runs are selected independently and can be chained differently. Instead of turn-run-turn-run as in the essay, the zebrafish can have turn-turn or run-run patters. Zebrafish turn direction is also correlated, as opposed to the random walk’s turn independence. A zebrafish left turn is more likely to follow a left turn.
When encountering darkness, the same-direction turns increase. When encountering light, alternating turns increase. Together with the sharp turn (O-bend) followed b shallower turns, this behavior should create a spiral-like search for the light area.
Note: this specialized circuitry in the hindbrain suggests that random search is a primitive behavior. Although the essay put the Levy walk logic in the midbrain, it belongs in the more primitive hindbrain.
Head direction
[Petrucco et al 2023] report that M.ip is highly connected with head direction axons from B.dtg (dorsal tegmental area of Gudden). This head direction does not receive vestibular input, but is likely derived from motor efferent copies. Since both M.ip and B.dtg are r1-derived regions and possibly ante-vestibular, this head-direction and M.ip connection may be ancient.
Speculative M.ip circuit following the fruit fly fan-shaped body. B.dtg dorsal tegmental nucleus of Gudde, B.rs reticulospinal motor command, Hb.m medial habenula, M.ip interpeduncular nucleus.
This organization is strikingly similar to the fruit fly’s ellipsoid body (EB), protocerebral bridge (PB), and fan-shaped body (FB) in the central complex (CX) [Hulse et al 2021]. EB and PB calculate head direction. The fan-shaped body merges head direction with goal direction to produce motor commands. In this diagram, M.ip represented as if it resembles the fan-shaped body.
If the M.ip functionality is similar to the fan-shaped body, it’s highly likely to be convergent evolution, not homology because amphioxus lacks any similar structure.
Dark search
When zebrafish are plunged into darkness, they initiate a search that continues for about five minutes. The darkness behavior increases speed and straight behavior [Horstick et al 2017]. In other words, phototaxis is not just gradient behavior but also has steady-state darkness behavior. Because zebrafish require light to hunt, darkness in itself is is an area to avoid.
The essay is purely gradient based and has no speed changes. Photokinesis is moving faster in darkness and slower in light, which will bias the time spent in the light area.
Although the previous essays have focused on bilateral locomotion in the style of Braitenberg machines, the chimaera brain hypothesis [Tosches and Adrendt 2013] suggests a distinct apical form of locomotion. The chimaera brain hypothesis suggests that bilaterian brains are the merger of an apical nervous system from the ancestral zooplankton larva state and a blastoporal (bilateral) nervous system from paired muscles along the spinal cord. The apical area contains unpaired light and chemical sensors and the blastoporal area contains bilateral, topographic somatic sense like touch. Apical navigation require a temporal gradient, calculated by sequential sampling because the apical senses are non-directional.
Apical unpaired light sensors and bilateral paired touch sensors in the chimaera brain.
The essays’ simulation scenario is a temporal phototaxis based on the real-time place preference (RTPP) experiments of [Chen et al 2014], where a zebrafish stayed inside a virtual light circle, avoiding surrounding dark area. Temporal phototaxis is gradient-based locomotion, heading toward light and away from darkness by comparing light samples at different times.
Chimaera locomotor
The vertebrate paired eye and paired olfaction are late vertebrate developments. The pre-vertebrate animal amphioxus has only a single frontal eye. Since pre-paired sense animals needed to navigate toward opportunities and away from threats, it’s conceivable that apical random-walk navigation developed before visual navigation.
Dual navigation system based on both temporal gradient-based random walks and bilateral spatial gradient navigation.
Apical gradient
The apical area contains undirected light and chemical sensors. The apical area is based on the zooplankton larval state as shown below, while the bilateral area is based on bilateral worm-like adults structured like the spinal cord of paired muscles and neurons.
Apical zooplankton larva. Note the single sensor on top.
Apical navigation requires following a temporal gradient, calculated by sequential sampling. While bilateral areas can compare left and right senses to calculate a spatial gradient, the single apical sense is restricted to a temporal gradient.
Bacteria tumble and run
Even simple bacteria can follow gradients using a directed random walk strategy called tumble and run [Segall et al 1986]. The bacteria’s flagella have two modes: tumble, turning without moving, and run, moving forward without turning. By alternating tumble with run, the bacteria can search with a random walk. By extending the run phase when the gradient improves, the bacteria can move toward the target.
Importantly, a temporal gradient calculation needs some sort of memory, accumulator or integrator to compare the current value to recent values. In bacteria, tis accumulator is an internal chemical quantity.
Different turning behavior depending on the gradient. Sharp turns moving into darkness and straight movement into light.
Apical and bilateral sensors
Amphioxus is a pre-vertebrate chordate that’s studied to understand vertebrate evolution. Amphioxus does not have a paired eye, instead it has a single frontal eye that amphioxus uses to orient vertically, and also a pineal-like photoreceptor [Lacalli 2020].
The pineal region is near the vertebrate habenula. Amphioxus does not have a habenula, but it does have a nearby motor control neuron, LPN3, with similar genetic markers [Bozzo et al 2023]. Modern vertebrate medial habenula receives undirected light input from the retina, but it seems plausible that an early habenula used the pineal photosensor because both are part of the same epithalamus complex, and only later connected to the newly-paired retina when it developed.
Multiple apical regions
Before diving into the vertebrate areas for apical locomotion, I need to explain why widespread areas can all be apical, including midbrain and hindbrain areas. Vertebrates had two rounds of whole genome duplication [Dehal and Moore 2005], which gives an easy evolutionary opportunity for four apical areas from the genome duplication, in addition to other possible duplications. The xenobot experiments [Blackiston et al 2023] shows that biology can mix multiple copies like the split apical area into a coherent animal: development can be flexible.
Functional vertebrate brain model showing apical areas as marked by fgf8 development transcription factor.
The diagram above is a functional representation of the vertebrate brain with possible apical areas highlighted in blue. Fgf8 is a development growth factor associated with the apical area [Marlow et al 2014]. The four (or five) possible apical area are as follows:
Prefrontal cortex and olfactory bulb.
The caudal isthmus area (r1) at the midbrain-hindbrain boundary, including cerebellum (CB), interpeduncular nucleus (M.ip), midbrain locomotor (MLR, M.ppt, M.ldt), head direction (B.dtg), parabrachial (B.pb), and part of the substantia nigra (Snr).
Habenula (Hb) and pre-thalamic eminence area (P.em) near ZLI. Also, between the septum-diagonal band (P.msdb) and preoptic area (Poa) near the optic (retina) region (R).
The mammilary and supramammilary area of the hypothalamus (H.mb and H.sum).
I’ve listed four regions instead of the five blue areas because the Hb-P.em-P.msdb area are physical closer than the diagram suggests, are split more by the alar/basal division than distance, and is complicated by distortions from the paired optic region.
This essay uses functions from the r1 isthmus area (M.ip and MLR.α / M.ldt), and from the habenula / retinal area (Hb.m, P.em, pineal, and retina). The logic behind their connectivity is a function split-and-pull like taffy or continental drift from a single pre-duplication area, for example, the isthmus apical area duplicating from an original pre-hypothalamus / retinal apical area.
As a note, the supramammilary area (H.sum) is highly connected with the area in this essay, but I’m postponing exploring its functionality for now.
Vertebrate apical and bilateral locomotor
Previously the essays used the bilateral locomotor path, going through the Vta (posterior tuberculum), tectum (OT), and midbrain locomotive region (MLR). The apical path runs through the medial habenula (Hb.m) and the interpeduncular nucleus (M.ip) before reaching the motor neurons.
Bilateral and apical locomotive paths. B.ll lateral line, B.rs reticulospinal motor neurons, B.ss somatosensory, Hb.m medial habenula, M.ip interpeduncular nucleus, MLR midbrain locomotive region, OT optic tectum, V.mr median raphe, 5HT serotonin.
The lamprey’s Hb.m supports locomotion for light, odor, and the lateral line [Stephenson-Jones et al 2012]. The lateral line is an aquatic sense for water flow, which allows fish to sense nearby objects. Although the exact functional division of phototaxis isn’t known, Hb.m, M.ip and the serotonin raphe nuclei (V.mr – 5HT) are all required [Cheng et al 2016].
For the essay’s simulation, I’m asking the integration and running average to the V.mr and 5HT, but this is something of a guess, because phototaxis integration hasn’t been measured. In the essay’s model, M.ip translates the light data and 5HT average into a gradient and into action.
Phototaxis actions
When zebrafish enter darkness from light, they immediately produce a large turn (O-bend) and start an area restricted search (ARS) [Fernandes et al 2012]. Later turns are smaller [Chen and Engert 2014].
For the essay, phototaxis gradient modifies the standard random walk. When entering darkness, M.ip increases the turn angle. When entering light, M.ip increases forward movement, extending the run.
Essay simulation showing the animal returning to light from darkness. The graph shows the gradient change when crossing the darkness boundary.
The above screenshot shows the animal crossing into darkness and returning to light. In the graph, “light” is the current photosensor value, “avg” is the running average measured by serotonin neurons, and “grad” is the different between the two. A gradient drop triggers high angle turns. A gradient rise triggers straight movement.
As the heat map in the right shows, this simple system produces real-time place avoidance (RTPA) of darkness. Since the system has no learning, there’s no conditioned place aversion (CPA).
Prethalamic eminence
The full phototaxis circuit in vertebrates is a bit more complicated because light input does through an intermediate area called the pre-thalamic eminence (P.em), which is between the habenula, hypothalamus and thalamus, and it one of the apical areas. Although P.em is not cortical, it provides neurons necessary for cortical development (Cajal-Retzius neurons for L1 patterning) [Marin-Padilla 2015] and neurons for habenula input, the habenula-projecting pallidum (P.hb) [Stephenson-Jones 2016].
Apical navigation paths, simplified in lamprey and extended in mammals. E.hc hippocampus, Hb.m medial habenula, M.ip interpeduncular nucleus, MLR midbrain locomotive region, Ob olfactory bulb, P.em pre-thalamic eminence, P.hb habenula projecting pallidum, R.gc retina ganglion cells.
Retina input goes through P.em to Hb.m for phototaxis. Interestingly, the main input for Hb.m in mammals is via cells that migrate from P.em and become the posterior septum (P.ps) [Watanabe et al 2018], which receives almost all of its input from the hippocampus (E.hc). If the hippocampus is an odor-processing system, then the olfactory bulb (Ob) to E.hc to Hb.m path is a chemotaxis path matching the retina’s phototaxis path.
Note that the olfactory placed develops from the lens placed and is differentiated by fgf8 [Bailey et al 2006]. So, it’s pleasing that the similar olfactory and hippocampal paths to Hb.m is a are chemotaxis and phototaxis paths split from a common ancestor.
Speculation
Although the lateral and medial habenula are chemically, connectional, and developmentally distinct, their broad similarity is interesting. If the medial habenula supports direct, concrete sensory navigation by gradient descent, perhaps the medial habenula supports more abstract value-based navigation for more abstract goals.
I like the model of motivation where the hypothalamus and midbrain structures like the ventral tegmental area (Vta), periaqueductal gray (M.pag), and parabrachial nucleus (B.pb) form the motivational core, primarily run by neuropeptides signaling, based on old chemical communication. [Damasio and Carvalho 2013] consider this area as an organized map for feelings, like the optic tectum (OT) has a retina-centric map for visual interest.
To avoid getting stuck in philosophical woo, I’m avoiding the question of whether this area is a primary source of feelings, but I like the idea of a semi-organized map at the base of motivation. The parabrachial nucleus (B.pb) is a good place to start, because its neurons encode warnings like pain, visceral summaries, and primitive feeding, including basic taste.
Parabrachial nucleus
B.pb provides a coarse summary of taste, pain, temperature, and visceral feelings like malaise without the details. It can report that something tastes good because it’s sweet or tastes bad because it’s bitter, but can’t experience chocolate. It’s more of an action-focused alarm [Campos et al. 2018] than a sensory experience.
For example, if B.pb detects bitter taste or malaise, it sends a general notice to other areas in the peptide core to stop eating and investigate further. If B.pb tastes sweet, it encourages eating. In addition to senses like taste and warning, B.pb has action control of its own, including reflexive escape actions, breathing and heart rate to the medulla (B.mdd) and B.nts. So, it can serve as a lower-level action hub.
B.pb and neuropeptides
B.pb poses an immediately difficulty for the simulation animal because it’s organized chemically by neuropeptides instead of a simple topological and connectivity map. The following diagram is a broad topographic map of B.pb [Chiang et al. 2019] that illustrates the issue.
Topological map of the parabrachial nucleus.
As shown above, the colored areas do not respect the named boundaries. The blue area represents taste neuron areas and the red area represents general alarm (pain, heat, cold, malaise, etc.) But even those colored areas are an oversimplification because neuron functions are mixed together salt-and-pepper style. [Pauli et al. 2022] found 21 subclusters of B.pb neuron peptide receptors and transmission, each of which may have distinct projection patterns.
This neuropeptide focus isn’t restricted to B.pb. The lateral hypothalamus (H.l), another major node in the feeding circuit, is also organized by neuropeptides, including important ones like orexin (exploring), and MCH, which it sends across the entire brain. Although [Diaz et al. 2023] has broken H.l into 9 areas, these may not be sufficient because of the neuropeptide focus. [Mickelsen et al. 2019] found 15 clusters of glutamate neurons and 15 clusters of GABA neurons. [Guillaumin and Burdakov 2017] and [Burdakov and Karnani 2020] find H.l functional communication through neuropeptides that are invisible to traditional synaptic communication.
Neuropeptide core
An “isodendritical core” [Ramón-Moliner and Nauta 1966] in the hypothalamus and midbrain is an old idea with a more modern description in [Agnati et al. 2010], which is a good starting point for the essay simulation. The core includes reticular areas of the hypothalamus, B.pb, M.pag, and the Vta (aka posterior tuberculum in zebrafish). “Neuropeptide core” matches my imagination of this area better than the old name. A diagram of the core is below, with the caveat that neuropeptide broadcasting is more important for communication than the diagram’s arrows.
Neuropeptide core in bright colors, associated areas greyed out. B.pb parabrachial nucleus, B.rs reticulospinal motor command, H.l lateral hypothalamus, H.pstn parasubthalamic nucleus, H.pv periventricular nucleus, H.vm ventromedial hypothalamus, Hb.l lateral habenula, Hb.m medial habenula, M.ip interpeduncular nucleus, M.pag periaqueductal, MLR midbrain locomotor region, OT optic tectum, P.bst bed nucleus of the stria terminals, S.a central amygdala, V.dr dorsal raphe (serotonin), V.mr medial raphe, Vta ventral tegmental nucleus (dopamine).
As shown above, the neuropeptide core is highly interconnected. B.pb includes taste and visceral sensation like nausea together with visceral control. H.l includes blood sensors like glucose level, insulin, and fat and protein levels. M.pag includes many innate behaviors including freezing, flight and grooming. Vta controls actions, including seeking and searching.
As the diagram illustrates, the neural connectivity of the inner core is not particularly useful because they’re all entirely interconnected. For simplicity of the essay simulation, I’m using a model where the core neuropeptides are shared in a common neuropeptide soup, or canal, where the neuropeptide identity is more important than the neuron’s specific physical location. For example, treating B.pb as one or two areas instead of the seven areas above.
Cerebrospinal fluid as neuropeptide canal
The periventricular areas like H.pv and M.pag are named for their location around the ventricles, which contains cerebrospinal fluid (CSF). These areas contain neurons that directly sense neurotransmitters and neuropeptides in the CSF itself. The CSF can be a canal for transmitting neuropeptides [Bjorefeldt et al. 2018].
Earlier photo-vertebrate animals may have used a similar canal more extensively. Because of the smaller brain size, diffusion in the canal may have been sufficient for communication without point to point synapses. [Vigh et al. 2004] point out that amphioxus larva, a pre-vertebrate chordate, has much of its com munition in a single neuropile (intertwined dendrites and axons) that’s open to sea water until its neural tube closes as an adult.
Neuropeptides and timing
Neuropeptides act on a much slower timescale than faster neurotransmitters like glutamate and GABA. Glutamate and GABA synapse are a few microseconds and clear rapidly. Neuropeptides can persist tens of seconds to tens of minutes. For an animal’s motivation, like fleeing a predator, the longer timescale is more appropriate, because the animal shouldn’t stop fleeing if it loses sight of the predator for ten milliseconds or even a second or two. Similarly, foraging for food is a longer task measured in many minutes or hours, not milliseconds. The longer chemical timing of the peptides is more suited to motivational timing than the fast reactive transmitters.
Peptide circuits
I’ve sketched out some possible neuropeptide circuits for feeding are portrayed in the diagram below, organized by behavior.
Sketch of some of the neuropeptide circuits related to feeding.
The first diagram shows dopamine as a primary seeking neurotransmitter [Alcaro et al. 2007]. When the animal finds target by odor in the simulation, dopamine tells the Vta to connect the olfactory bulb (Ob) to the motor locomotive region (MLR), aiming the animal to the food scent.
The second shows orexin as a general food exploration signal. In contrast with the target-focused seeking, exploration is a random search.
The third is part of the eating circuit, where CGRP (an alarm neuropeptide) tonically inhibits eating, until AgRP (a hunger neuropeptide) disinhibits it [Essner et al. 2017].
Chiang, M. C., Bowen, A., Schier, L. A., Tupone, D., Uddin, O., & Heinricher, M. M. (2019). Parabrachial Complex: A Hub for Pain and Aversion. The Journal of neuroscience : the official journal of the Society for Neuroscience, 39(42), 8225-8230.
The essay 22 simulation explored a striatum model where the two decision paths competed: odor seeking vs random exploration, using dopamine to bias between exploration and seeking. This model resembled striatum theories like [Bariselli et al. 2020] that consider the stratum’s direct and indirect paths as competing between approach and avoidant actions.
Issues in essay 22 include both neuroscience divergence and simulation problems. Although the simulation is a loose functional model, that laxity isn’t infinite and it may have gone too far from the neuroscience.
Adenosine and perseveration
Seeking and foraging have a perseveration problem: the animal must eventually give up on a failed cue, or it will remain stuck forever. The give-up circuit in essay 22 uses the lateral habenula (Hb.l) to integrate search time until it reaches a threshold to give up. An alternative circuit in the stratum itself involves the indirect path (S.d2), the D2 dopamine receptor and adenosine, with a behaviorally relevant time scale.
When fast neurotransmitters are on the order of 10 milliseconds, creating a timeout on the order of a few minutes is a challenge. Two possible solutions in that timescale are long term potentiation (LTP) where “long” means about 20 minutes, and astrocyte calcium accumulation, which is also about 10 to 20 minutes.
Adenosine receptors (A2r) in the striatum indirect path (S.d2) measure broad neural activity from ATP byproducts that accumulate in the intercellular space. Over 10 minutes those A2r can produce internal calcium ion (Ca) in the astrocytes or via LTP to enhance the indirect path. Enhancing the indirect path (exploration), eventually causes a switch from the direct path (seeking) to exploration, essentially giving-up on the seeking.
Ventral striatum
Although the essay models the dorsal striatum (S.d), the ventral striatum (S.v aka nucleus accumbens) is more associated with exploration and food seeking. In particularly, the olfactory path for food seeking goes through S.v, while midbrain motor actions use S.d. In salamanders, the striatum only processes midbrain (“collo-“) thalamic inputs, while olfactory and direct senses (“lemno-“) go to the cortex [Butler 2008]. Assuming the salamander path is more primitive, the essay’s use of S.d in the model is a likely mistake.
But S.v raises a new issue because S.v doesn’t use the subthalamus (H.stn) [Humphries and Prescott 2009]. Although, that model only applies to the S.v shell (S.sh) not the S.v core (S.core).
In the above diagram of a striatum shell circuit, an odor-seek path is possible through the ventral tegmental area (Vta) but there is no space for an alternate explore path.
Low dopamine and perseveration
[Rutledge et al. 2009] investigates dopamine in the context of Parkinson’s disease (PD), which exhibits perseveration as a symptom. In contrast to the essay, PD is a low dopamine condition, and adding dopamine resolves the perseveration. But that resolve is the opposite of essay 22’s dopamine model, where low dopamine resolved perseveration.
Now, it’s possible that give-up perseveration and Parkinson’s perseveration are two different symptoms, or it’s possible that the complete absence of dopamine differs from low tonic dopamine, but in either case, the essay 22 model is too simple to explain the striatum’s dopamine use.
Dopamine burst vs tonic
Dopamine in the striatum has two modes: burst and tonic. Essay 22 uses a tonic dopamine, not phasic. The striatum uses phasic dopamine to switch attention to orient to a new salient stimulus. The phasic dopamine circuit is more complicated than the tonic system because it requires coordination with acetylcholine (ACh) from the midbrain laterodorsal tegmentum (V.ldt) and pedunculopontine (V.ppt) nuclei.
A question for the essays is whether that phasic burst is primitive to the striatum, or a later addition, possibly adding an interrupt for orientation to an earlier non-interruptible striatum.
Explore semantics
The word “explore” is used differently by behavioral ecology and in reinforcement learning, despite both using foraging-like tasks. These essays have been using explore in the behavioral ecology meaning, which may cause confusion on the reinforcement learning sense. The different centers on a fixed strategy (policy) compared with changing strategies.
In behavioral ecology, foraging is literal foraging, animals browsing or hunting in a place and moving on (giving up) if the place doesn’t have food [Owen-Smith et al. 2010]. “Exploring” is moving on from an unproductive place, but the policy (strategy) remains constant because moving on is part of the strategy. The policy for when to stay and when to go [Headon et al. 1982] often follows the marginal value theorem [Charnov 1976], which specifies when the animal should move on.
In contract, reinforcement learning (RL) uses “explore” to mean changing the policy (strategy). For example, in a two-armed bandit situation (two slot machines), the RL policy is either using machine A or using machine B, or a fixed probabilistic ratio, not a timeout and give-up policy. In that context, exploring means changing the policy not merely switching machines.
[Kacelnick et al. 2011] points out that the two-choice economic model doesn’t match vertebrate animal behavior, because vertebrates use an accept-reject decision [Cisek and Hayden 2022]. So, while the two-armed bandit may be useful in economics, it’s not a natural decision model for vertebrates.
Avoidance (nicotinic receptors in M.ip)
The simulation uncovered a foraging problem, where the animal remained around an odor patch it had given up on, because the give-up strategy reverts to random search. Instead, the animal should leave the current place and only resume search when its far away.
Path of simulated animal after giving up on a food odor.
In the diagram above, the animal remains near the abandoned food odor. The tight circles are the earlier seek before giving up, and the random path afterwards is the continued search. A better strategy would leave the green odor plume and explore other areas of the space.
As a possible circuit, the habenula (Hb.m) projects to the interpeduncular nucleus (M.ip) uses both glutamate and ACh as neurotransmitters, where ACh amplifies neural output. For low signals without ACh, the animal approaches the object, but high signals with ACh switch approach to avoidance. This avoidance switching is managed by the nicotine receptor (each) which is studied for nicotine addiction [Lee et al. 2019].
An interesting future essay might explore using nicotinic aversion to improve foraging by leaving an abandoned odor plume.
After essay 21 changed the animal’s default movement to a Lévy exploration, it’s immediate to ask whether that random search is a full action, just like a seek turn or an avoid turn. An if exploration is a controlled action, then the model needs to treat exploration as a full action, like approach or avoid.
Exploration as a full locomotive system at the level of approach and avoid.
[Cisek 2020] identifies a vertebrate system for exploration, including the hippocampus (E.hc) and its associated nuclei such as the retromammilary hypothalamus (H.rm aka supramammilary). Essay 22 considers the idea of treating the subthalamic nucleus (H.stn) as part of the exploration circuit.
Subthalamic nucleus
H.stn is a hypothalamic nucleus from the same area as H.rm, which is part of the hippocampal theta circuit, which synchronizes exploration and spatial memory and learning. However, H.stn is part of the basal ganglia and not directly connected with the exploration system.
[Watson et al. 2021] finds a locomotive function of H.stn, where specific stimulation by the parafascicular thalamus (T.pf) to H.stn starts locomotion. If the stimulation is one-sided, the animal moves forward with a wide turn to the contralateral side. T.pf includes efference copies of motor actions from the MLR as well as from other midbrain actions.
Locomotion induced in the H.stn by T.pf stimulation. H.stn sub thalamic nucleus, T.pf parafascicular nucleus, MLR midbrain locomotor region.
For essay 22, let’s consider the H.stn locomotion as exploration. Since H.stn is part of the basal ganglia, the bulk of essay 22 is considering how exploration might fit into the proto-striatum model of essay 18.
Striatal attention and persistence
Since the current essay simulation animal is an early Cambrian proto-vertebrate, it doesn’t have a full basal ganglia. Evolutionarily, the full basal ganglia architecture could not have sprung into being fully formed; it must have developed in smaller step. Following a hypothetical evolutionary path, the essays are only implementing a simplified striatal model, adding features step-by-step. Unfortunately, because there’s no living species with a partial basal ganglia — all vertebrates have the full system — the essay’s steps are pure invention.
The initial striatum of essay 18 was a partial solution to a simulation problem: persistence. When the animal hit a wall head on, activating both touch sensors, it would choose randomly left or right, but because the simulation is real-time not turn-based, at the next tick both sensors remained active and the animal would choose randomly again, jittering at the wall until enough turns of the same direction escaped the barrier.
Proto-striatum for persistence by attention. Action feedback biases the choice to the last option: win-stay. B.rs reticulospinal motor command, Ob olfactory bulb, MLR midbrain locomotor region, Snc substantia nigra pars compacta (posterior tuberculum).
The main sense-to-action path is from the olfactory bulb (O.b) through the substantia nigra (Snc aka posterior tuberculum in zebrafish) to the midbrain locomotor region (MLR) and to the reticulospinal motor command neurons (B.rs), following the tracing and locomotive study of [Derjean et al. 2010] in zebrafish and Vta/Snc control of locomotion in [Ryczko et al. 2017]. The proto-striatum circuit is built around that olfactory-seeking circuit, acting persistent attention.
The proto-striatal model uses an efference copy of the last action from the MLR to bias the choice of the next action via a MLR to T.pf to striatum path. The model biases the choice through removing inhibition of the odor to action path. If the last action as left, the left odor is disinhibited, making it more likely to win.
The striatal system uses disinhibition for noise reasons. [Cohen et al. 2009] studied attention in the visual system and found that attention removed coherent noise by removing inhibition. By removing inhibition, the attended circuit is less affected by the controlling circuit’s noise.
Note: essay 19 considered an alternative solution to the attention issue by following the nucleus isthmi system in zebrafish as studied in [Grubert et al. 2006], where the attention to the win-stay odor used acetylcholine (ACh) amplification to bias the choice.
Striatal columns: approach and avoid
An immediate difficulty with the simple proto-striatal model is the lack of priority. Although left vs right have equal priority, avoiding a predator is more important than seeking a potential food source. Unfortunately, the proto-striatum treats all options equally. As a solution, essay 18 split the striatum into columns, where each column resolves an internal conflict without priority (“within-system”) and the columns are compared separately (“between-systems”), where “within-system” and “between-system” are from [Cisek 2019].
Dual striatum column for approach and avoid, where MLR resolves the final conflict. B.rs reticulospinal command neuron, B.ss somatosensory (touch), MLR midbrain locomotive region, M.pag periaqueductal gray, Ob olfactory bulb, S.ot olfactory tubercle,S.d dorsal striatum.
Subthalamic nucleus and exploration
If we now treat exploration as a distinct action system, then it needs its own control system and column in the proto-striatum. The within-system choice for exploration is the left and right turns for a random walk, and the between-system choices are between the exploration system and the odor-seeking system.
As a possible neural correlate of exploration, consider the sub thalamic nucleus (H.stn). The sub thalamic nucleus is derived from the hypothalamus, specifically from the same area as the retromammilary area (H.rm aka supramammilary), which is highly correlated with hippocamptal theta, locomotion and exploration.
[Watson et al. 2021] finds a locomotive function of H.stn, where specific stimulation by the parafascicular thalamus (T.pf) produces locomotion via the midbrain locomotive region (MLR). T.pf includes efference copies of motor actions from the MLR as well as other midbrain action efference copies. In the proto-striatum model, the feedback from MLR to striatum uses T.pf.
Exploration locomotive path through H.stn. H.stn sub thalamicnucleus, MLR midbrain locomotive region, T.pf parafascicular thalamus.
Seek and explore with dual striatal columns
Suppose the striatum manages both odor seeking (chemotaxis) and default exploration (Lévy walk). The two actions are conflicting with a complex priority system. When a food odor first appears, the animal should seek toward it (priority to seek), but if no food exists the animal should resume exploration (priority to explore). To resolve the between-system conflict, the two strategies need to columns with lateral inhibition to ensure that only one is selected.
Dual striatum columns for seek and explore strategies. B.rs reticulospinal motor command, H.stn sub thalamic nucleus, Ob olfactory bulb, P.ge globus pallidus external, S.d1 direct striatum projection, S.d2 indirect striatum projection, Snc substantia nigra pars compacta, Snr substantia nigra pars reticulata.
Selecting the seek column enables the odor sense to MLR path, seeking the potential food odor. Selecting the explore column enables the H.stn to MLR path, randomly searching for food.
Note: the double inversion in both paths is to reduce neuron noise [Cohen et al. 2009]. Removing inhibition reduces noise, where adding excitation would add noise. In the essay stimulation, this double negation isn’t necessary.
Striatum with dopamine/habenula control
The previous dual column circuit isn’t sufficient for the problem, because it lacks a control signal to switch between exploit (seek) and explore. The striatum dopamine circuit might help this problem by bringing in the foraging implementation from essay 17.
A major problem in essay 17 was the tradeoff between persistence and perseverance in seeking an odor. Persistence ensures that seeking an odor will continue even when the intermittent. Perseverance is a failure mode where the animal never gives up, like a moth to a flame. As a model, consider using dopamine in the striatum as persistence or effort [Salamone et al. 2007], and control of dopamine by the habenula as solving perseverance with a give-up circuit.
Explore and exploit (seek) columns controlled by dopamine. H.l lateral hypothalamus, Hb.l lateral habenula, H.stn sub thalamic nucleus, MLR midbrain locomotive region, Ob olfactory bulb, P.em pre thalamic eminence, P.ge globus pallidus external, S.d1 striatum direct projection, S.d2 striatum indirect projection, Snc substantia nigra pars compacta, Snr substantia nigra pars reticulata.
The striatum uses two opposing dopamine receptors named D1 and D2. D1 is a stimulating modulator though a G.s protein path, and D2 is an inhibiting modulator through a G.i protein path. In the above diagram, high dopamine will activate the seek column via D1 and inhibiting the explore column via D2. Low dopamine inhibits the seek column and enables the explore column. So dopamine becomes an exploit vs explore controller.
In many primitive animals, dopamine is a food signal. In c.elegans the dopamine neuron is a food-detecting sensory neuron. In vertebrates, the hunger and food-seeking areas like the lateral hypothalamus (H.l) strongly influence midbrain dopamine neurons both directly and indirectly. Indirectly, H.l to lateral habenula (Hb.l) causes non-reward aversion [Lazaridis et al. 2019].
For the essay, I’m taking H.l as multiple roles (H.l is a composite area with at least nine sub-areas [Diaz et al. 2023]), both calculating potential reward (odor) via the H.l to Vta/Snc connection, and cost (exhaustion of seek task without success) via the H.l to Hb.l to Vta/Snc connection.
Lazaridis I, Tzortzi O, Weglage M, Märtin A, Xuan Y, Parent M, Johansson Y, Fuzik J, Fürth D, Fenno LE, Ramakrishnan C, Silberberg G, Deisseroth K, Carlén M, Meletis K. A hypothalamus-habenula circuit controls aversion. Mol Psychiatry. 2019 Sep
Previous essays used a simple default ballistic forward motion action: the animal moved forward until it hit an obstacle or encountered food or an odor plume. Furthermore, all actions in the essay were continuous: at every time-step direction or speed could change. The animal was slug-like. However, since vertebrate is oscillatory — swimming or walking — powered by central pattern generators (CPGs), vertebrate motor commands are not continuous but modifiable only on correct timing of the oscillatory cycle.
This essay 21 adds some more realism to the simulated animal by creating distinct action syllables, each of which runs to completion. In addition, the default movement is a Lévy walk to more efficiently search for food.
Zebrafish bouts and mouse modules
Zebrafish larva move in discrete bouts [Johnson et al. 2020], punctuated by pauses. Each bout is on the order of 200ms to 1000ms and consists of stereotyped movement like a forward swimming stroke, and several turn types possibly followed by forward motion. The neural source of the bout timing is not know, although basal ganglia (striatal) defects can also produce jerky motion instead of smoothly linked motion.
Mouse movement is also comprised of small modules [Wiltschko et al. 2016] on the order of 60 modules in open exploration, each module lasting 200ms to 500ms. Unlike the zebrafish larva, sequences of mouse modules are linked smoothly instead of in jerky bouts.
Simulation action syllables
To match the vertebrate action syllables, the essay simulation now has action syllables at the lowest layers. The action syllables are fixed programs that last for several simulation ticks. For example a forward left turn might take 10 simulation ticks, depending on the tick resolution. In other words, these syllables are more like real-time actions with limited time resolution, not turn-based actions like a board game.
Unlike the current action completes, the system ignores new neural commands. In the future, specialized commands like freeze or panic escape might override the current action, like fast escape of zebrafish that bypass normal motor logic. In theory the system could also incorporate mid-action modulation, such as adding power to a swimming stroke without altering the basic action.
The new syllables should add realism and also introduce complications that vertebrates need to solve, such as timing for sequences of syllables. The syllables also introduce issues because prior-action memory as modeled by the striatum and nucleus isthmi need to persist for a syllable, not just a simulation tick.
Lévy walks
Along with the syllable changes, essay 21 adds better default movement. Previously if the animal wasn’t approaching food or avoiding an obstacle, it would move forward ballistically. That simplification only worked because the simulation is a simple bounded box, but in nature animals have better default search strategies.
Brownian motion is a simple default. The animal turns a random direction, then moves forward a random (gaussian) distance. Brownian motion does well at searching a neighborhood and is better than the essay’s ballistic strategy, but it tends to get stuck in a small area. When food is scarce and patchy, the brownian strategy won’t move the long distances needed to effectively find a new patch. The Lévy walk improves on this strategy by moving long distances when the current neighborhood is already searched [Abe 2020].
Default random walk for the simulated animal.
A Lévy walk is fractal (“scale-free”): larger walks have the same search structure as neighborhood search. If described in probabilistic terms, the distance traveled looks like a power law:
P(len) = 1/ len ^ 2
Where the exponent 2 (alpha) can be generalized between 1 and 3. The bounds are because the exponent 1 or less is ballistic and exponent 3 or more is essentially brownian. If alpha is closer to 1, the search is wider and moving further, while an alpha is closer to 3, the search is closer and more local.
Central pattern generators, criticality, and chaos
Animals do seem to use Lévy walks [Kölzsch et al. 2015] and the source appears to be internally generated [Berni et al. 2012] as opposed to a response to a fractal environment. The internal source of the randomness is not well know, although central pattern generators appear to be a possibility [Sims et al. 2018], [Reynolds 2019].
Near a critical point, fractal patterns appear and can be efficient computationally [Abe 2020], and a relatively simple chaotic system with only two random variables and produce Lévy walks. More broadly, other brain areas such as the cortex may also use criticality or near criticality to improve computation and generate longer-lasting signals to solve the timing problem [Hidalgo et al. 2014]. The timing problem is that fast neurons are 10ms but behavior needs to be responsive on the order of seconds and minutes.
Vertebrate source of Lévy walks
Although there is some knowledge of the search circuitry in fruit flies [Berni et al. 2012], the vertebrate circuitry seems entirely unknown.
As a thought experiment, consider the midbrain theta circuitry as part of the exploration circuit [Cisek 2022], and therefore related to the Lévy walk. If the vertebrate search is generated by a combination of central pattern generators then the source should be in the hindbrain, near those CPGs.
Theta clock cycles (4-12hz) in the brain are strongly correlated with exploratory movement, thought, and learning especially with the hippocampus (E.hc).
In the above, the hindbrain contains the chaotic search circuitry and generates theta, such as B.pno and B.vtn in conjunction with the reticulospinal motor command (B.rs). B.vtn is the forward movement analogue of the head direction nucleus B.dtn.
The medial septum (P.ms) is essentially the main theta clock for the hippocampus. Part of the theta cycle is generated internally to P.ms with spontaneous interaction of acetylcholine (ACh) and GABA inhibitory neurons, but exploration-related theta comes from the hypothalamic retromammilary (H.rm, aka supramammilary), which receives its theta from the hindbrain theta centers.
The movement restriction for theta depends on the median raphe (V.mr), which is one of the two main serotonin (5HT) centers. If V.mr is disabled, theta through H.rm always exists, not restricted to exploration. V.mr is in turn strongly influenced with the habenula (Hb.m) and interpeduncular (M.ip) complex.
Simulation
For now, I’m avoiding using criticality or chaotic variables in the simulation, although criticality is an interesting design to explore and very possibly how the vertebrate brain solves these problems.
The disadvantage is that the simulation would quickly become unclear and overcomplicated. While a few chaotic variables in the brainstem might be manageable, extending that idea to the striatum and cortex seems like it would become impenetrable. Since the purpose of the simulation is something like an executable thought experiment or executable diagram, and impenetrable simulation is defeating the purpose. Some alternatives like [Bartumeus and Levin 2008] fractal reorientation clocks might serve the purpose while remaining more clear.
Although the essays have implemented obstacle avoidance, they haven’t yet explored olfactory avoidance. Olfactory avoidance is distinct from obstacles, not just because obstacles have higher priority, but because the olfactory system is from an entirely different nervous system than the sensorimotor system. In the chimaeral brain theory [Tosches and Arendt 2013], bilaterian brains are composed of an apical nervous system (ANS) focused on chemo senses (olfactory external and hypothalamic internal), and a blastoporal nervous system (BNS) focused on sensorimotor control like obstacle avoidance.
Olfactory path
The paths for olfactory motion compared with obstacle motion shows the value of the chimaeral theory in making sense of the brain. Working backward from the midbrain locomotive region (MLR), the acetylcholine (ACh) MLR nuclei specialize: the pedunculopontine nucleus (M.ppt) supports the sensorimotor BNS, and the laterodorsal tegmental nucleus (M.ldt) supports the chemosensory ANS.
Sensor-locomotion paths: olfactory on top and somatosensory on bottom. B.ll lateral line, B.rs reticulospinal motor command, B.ss somatosensory, Hb.m medial habenula, M.ldt laterodorsal tegmental nucleus, M.ppt pedunculopontine nucleus, Ob.m medial olfactory bulb, OT tectum, R.vis visual input, ,Vta ventral tegmental area.
In the above diagram, food odors and warning odors use distinct paths to the MLR. Food odors from the olfactory bulb (Ob) pass through the ventral tegmental area (Vta – posterior tuberculum in zebrafish) to the MLR [Derjean et al. 2010]. Aversive odors like cadaverine pass through the medial habenula (Hb.m) to the M.ldt portion of the MLR [Stephenson-Jones et al. 2012]. The food and avoidance paths are distinct because hunger and satiety from the hypothalamus modulate the food path, while the avoidance path can pass through unmodulated. These olfactory locomotion paths correspond to the ANS.
Lamprey medial habenula path
All vertebrates share this basic architecture, including the lamprey, one of the most evolutionary-distant vertebrates. [Stephenson-Jones et al. 2012] traced the Hb.m circuit, showing that Hb.m inputs are from the olfactory path, the parapineal (light attraction), and an electron-sensory alarm to the interpeduncular nucleus (M.ip).
Lamprey olfactory warning path through the habenula to the MLR. M.ip interpeduncular nucleus.
The above diagram fills out the olfactory warning path. The interpeduncular nucleus is a key node in the avoidance circuit, and also key to locomotor-induced theta, and one of the two serotonin nodes. Mip has a major output to the serotonin areas: dorsal raphe (V.dr) and medial raphe (V.mr) and to the central grey (M.pag) [Quina et al. 2017] and M.ldt as well as structures associated with hippocampal (E.hc) theta [Lima et al. 2017].
Medial habenula behavior
In larval zebrafish, Hb.m supports olfactory avoidance [Choi et al. 2017], [Jeong et al. 2021], and light seeking [Zhang et al. 2017]. At least one study indicates that it may also affect food seeking [Chen et al. 2019]. The non-Ob input to Hb.m — the posterior septum (P.ps) — produce locomotion when stimulated [Ostu et al. 2018], suggesting that later evolved functionality maintains the original basal function.
In zebrafish, M.ip only projects to serotonin areas (V.dr and V.mr), not to dopamine or MLR areas. The lamprey connectivity suggests that the M.ip to M.ldt connection was lost in fish.
The Hb.m to M.ip connection is affected by nicotine. An interesting property is that low stimulation and high stimulation have opposite effects. Low stimulation uses glutamate connections and is attractive while high stimulation adds ACh and is aversive [Krishnan et al. 2014].
Developmental genetic notes
As an interesting aside, both Hb.m and avoidant layers of OT shared a genetic marker Brn3a (aka pou4f1) [Quina et al. 2009], [Fedtsova et al. 2008]. That marker also appears in the cerebellum’s inferior olive, trigeminal sensory areas, and the amphioxus motor LPN3 neuron [Bozzo et al. 2023].
M.ldt and M.ppt are sibling areas, deriving from the r1 rhombic lip [Machold et al. 2011].
Glutamate and GABA neurons in M.ip, Vta, and M.ldt all derive from r1 basal neurons [Lahti et al. 2016].
Locomotion switchboard
The addition of olfactory avoidance further complicates the switchboard combining the various locomotor streams, especially if the olfactory path uses serotonin as a modulator as opposed to a straight glutamate connection. Although I’ll probably use a fixed priority for essay 20, and as [Cisek 2022] notes, avoidance can be combined additively, at some point the switchboard will need more control, especially when essays add vision and consummatory actions.
Essay 18 was trying to solve the problem of maintaining behavioral state. When a fast neuron synapse takes only 5ms, behavior that lasts seconds or minutes needs some circuit to sustain attention on the task. Essay 18 explored the striatum as a possible model to maintain behavior. In zebrafish, this problem is partial solved with a paired system consisting of the optic tectum (OT) and the nucleus isthmi (NI) [Gruberg et al. 2006].
Optic tectum
The optic tectum (OT – superior colliculus in mammals) is a midbrain action and sensor system that organizes vision, touch, sound, and action into retinotopic map like an air controller radar screen that activates only for important triggers. So, it’s not like the movie screen of primate vision, but is an action-oriented, sparse map that focuses on a few important items. In the larva zebrafish, the OT activates for hunting prey (paramecia) and avoiding obstacles and predators.
The OT itself has no persistence, When it detects potential prey and moves toward the prey, the OT doesn’t remember that it’s hunting or recall the previous location of the prey. Without enhancement, it forgets the pretend fails the hunt. The nucleus isthmi (NI – parabigeminal in mammals) provides that attention and persistence function [Henriques et al. 2019].
Nucleus isthmi circuit
The NI has a simple organization that is topologically, bidirectionally mapped to OT. The return signal from NI to OT is acetylcholine (ACh), which amplifies the sense input, biasing the next action to follow the previous action. Essentially this is a simple attention circuit that maintains consistent behavior.
Optic tectum and nucleus isthmi circuit as used in the essay 19 simulation.
In the diagram above, a left action sends an efference copy to the matching nucleus isthmi area, which can remember the activation for longer than the 5ms fast activation in the OT. In turn it sends an ACh modulator to amplify the left touch sensor, biasing the direction toward the same action.
For the essay simulation, the original problem was hitting an obstacle head-on, which triggered both left and right touch sensors, which then caused jitter as the animal randomly chose left and right without maintaining consistency. By adding an NI system, an initial left action would bias the left input sense to choose a next left action.
Acetylcholine attention system
As a speculation, or perhaps a mnemonic, this NI system where ACh enhances senses based on action might be a model for some attention mechanisms else were in the brain. NI is a sister nucleus to other ACh nuclei, specifically the parabrachial nucleus (B.pb) and the pedunculopontine nucleus (V.ppn), all developing from the same stem region near the isthmus. V.ppn is one of the major ACh attention nuclei and is part of the midbrain locomotive region (MLR). It seems plausible that V.ppn might share some organization with NI where its upstream ACh might support sense attention like the NI does for OT.
Engineering note
After implementing the nucleus isthmi support, both the proto-striatum and NI solve the jittering problem equally. The algorithms are slightly different — NI is a straight enhancement, while proto-striatum is a disinhibition with selection — but for the current complexity of the animal and environment, there’s no behavioral difference. Both proto-striatum and NI can be enabled simultaneously without interference problems.
The previous proto-striatum model is flawed because it focused too much on sensory input and not enough on action efferent copies. To fix this focus, the model can use midbrain locomotive region (MLR) actions as a bias selector.
Recall that the simulation needed the striatum to solve an action jitter problem by introducing a win-stay bias. Once the animal turns left, it should bias toward continued left turns. Before the fix, the animal randomly chose a direction every 50ms, reversing itself, causing problems in avoiding corners and obstacles. The simulation problem was an action-selection problem not a sensor problem.
In the vertebrate striatum, action feedback comes from the MLR via the parafascicular thalamus (T.pf). The T.pf connection to the striatum is unique, both in its targeting of striatal interneurons (S.cin and S.pv), but also for its connection to the medium spiny projection neurons (S.spn), the main striatal neurons [Ragu et al. 2006]. T.pf connects directly to S.spn dendrites, not merely the spines as with other inputs. This direct connection potentially gives a stronger stimulus, and its uniqueness suggests it may be an older, more primitive connection.
Action-focused striatum model
So, I’m changing the striatum model to follow an action focus. After an action fires the motor command neurons (B.rs reticulospinal), the MRL sends an efferent copy of the motor command to the striatum via T.pf.
Action feedback model for proto-striatum. B.rs reticulospinal motor command, MLR midbrain locomotive region, Ob olfactory bulb, Snc substantia nigra pars compacta, S.pv striatal parvalbumen interneuron, S.spn spiny projection neuron.
In the above diagram, the main sensor path is still from the olfactory bulb (Ob) to the substantia nigra pars compacta (Snc / posterior tuberculum) and then to MLR, basically a stimulus-response path. A previous action biases the sensory path for the next action by activating a corresponding S.spn, which disinhibits Snc, making the next sensory input more powerful.
Comparison with the previous model
As a comparison, the following diagram shows the previous striatal model. Unlike the new model, the final selected action didn’t bias the next action because there was no feedback connection. (The reset signal to S.pv is a different circuit, and doesn’t bias the decision because it applies to all choices equally.)
Previous photo-striatum, where a prior selected sense biased the next sense. B.ss somatosensory touch.
In addition, the sensory input must coordinate striatal disinhibition via S.spn with its excitation of the Snc action. Although not impossible evolutionarily, the double coordination required makes it less likely. The new model not only incorporates the action but simplifies the sensor circuit.
Parafascicular thalamus
For personal reference, here’s a summary of the T.pf connections [Smith et al. 2022].
Connections of the parafascicular thalamus.
Essentially all the T.pf inputs are motor efference copies and all the T.pf outputs are to the basal ganglia. Inputs include the following areas: vision/optic motor (OT and pretectum), midbrain locomotive region (MLR, M.pag, V.ppt, V.ldt), diencephalon locomotive region (H.zi), consummatory action (B.bp), forebrain attention (P.bf) and cortical action (C.fef, C.moss, C.gu). The cingulate cortex might be unusual (C.cc), although it also has motor areas.
Striatum as attention
Attention is a difficult topic, in part because it’s used in so many diverse ways that the word is often more confusing than helpful [Hommel et al. 2019], [Krauzlis et al. 2014]. However, I think it’s interesting that the action-based striatum model looks like selective attention.
Simplification of proto-striatum showing resemblance to selective attention.
When a left action biases the next action to stay the same, its mechanism is to enhance the sensory path, as if it’s paying attention more to one side than another.
Engineering feedback: dopamine mistake
When implementing this idea, the simulation doesn’t need dopamine feedback. Instead of forcing the dopamine just because the basal ganglia has dopamine feedback I’m taking it out from the model. Since I’ve only implemented a prototype portion of the basal ganglia, this may be okay instead of a fatal flaw. When the full model arises, we’ll see if this is a mistake.
Actual simulation implementation, removing dopamine and reset feedback.
Notice that the only dopamine in this model is descending, with no ascending dopamine [Ryczko and Dubuc 2017].
References
Hommel B, Chapman CS, Cisek P, Neyedli HF, Song JH, Welsh TN. No one knows what attention is. Atten Percept Psychophys. 2019 Oct
The planned striatum model of essay 17 quickly runs into simulation problems because it’s missing priority selection between avoiding obstacles and seeking food. Obstacle avoidance needs a higher priority than seeking an odor plume, but a naive striatum doesn’t support that priority.
Broken striatum model where toward and away have no priority. Ob olfactory bulb, B.ss somatosensory touch, B.rs reticulospinal motor command.
This model fails because this striatum has no priority of away (avoid) actions from toward (approach) actions. An animal can’t simply follow an odor blindly, ignoring obstacles, but this model doesn’t support that priority.
Tectum
Adding the tectum seems like the right solution, although I was planning on putting it off until dealing with vision.
The tectum (optic tectum / superior colliculus) is better known for its vision support, but the deeper tectum layers are a general action-decision system. At its lower levels near periaqueductal gray (M.pag) it has a topographic direction-based map on its intermediate level and an action-based map in the deep level.
The tectum and M.pag are neighbors, almost layers of each other, and in animals like the frog, the M.pag is as a deeper layer of the tectum.
Relation between M.pag and OT in mammals (left) and frog (right), where the ventricle shape determines the anatomical label for homologous areas.
The tectum is an action organizer, not just a vision organizer. For the simulation, the action matters since the simulated animal doesn’t have vision.
Amphioxus, a non-vertebrate chordate that’s a model into pre-vertebrate evolution, has a few motor-related cells with the same genetic markers as the tectum [Pergner et al. 2020]. It’s conceivable that the amphioxus tectum is more action focused, since the amphioxus frontal eye is only a dozen photoreceptors with no lens.
Action categories
The tectum has split circuits for turning and for approach and avoid [Wheatcroft et al. 2022]. The simulation can use something like the following circuit.
Split tectum and striatum circuit. B.rs reticulospinal motor command, B.ss somatosensory input, M.lr midbrain locomotor region, M.pag periaqueductal gray, Ob olfactory bulb, S.d dorsal striatum, S.ot olfactory tubercle.
Approach (toward) senses like food odors excited toward actions, and avoidant (away) sense like touch excite away actions. Because the priority areas are split, each striatum can choose between non-priority options (left vs right). The priority resolves only later in the midbrain locomotor region, using context input to decide which major direction to use. In this split model, the simplified striatum circuit can work because all of striatum options are equal priority.
As a note on accuracy, the diagram misrepresents the actual olfactory path, specifically the real olfactory tubercle. In reality, olfaction has a distant, complicated path to the tectum.
Short-cut escape signal
The previous diagram is also misleading because it’s too organized, as if each function has a dedicated, planned circuit. Although the tectum itself is highly-organized, the downstream and modulating circuits are more ad hoc. For example, the zebrafish has an escape mechanism that short-cuts the tectum and drives the B.rs command motor directly [Zwaka et al. 2022].
Fast escape shortcut of tectum-mediated locomotion.
In the above diagram, the escape circuit short-circuits any decisions of the tectum and striatum. Relatedly, the “switch” area in M.lr isn’t as tidy as the diagram suggests. It’s more like that M.lr contains multiple actions which laterally inhibit each other in a priority scheme, modulated by M.pag.
As an additional correct, many of the modulators like M.pag affect the tectum directly, instead of the diagram’s dedicated priority-resolution function.
