Generalization of the resource-rationality principle to neural control of goal-directed movements.

We review evidence that the resource-rationality principle generalizes to human movement control. Optimization of the use of limited neurocomputational resources is described by the inclusion of the "neurocomputational cost" of sensory information processing and decision making in the optimality criterion of movement control. A resulting tendency to decrease this cost can account for various phenomena observed during goal-directed movements.

Hyperthermia induced by transient receptor potential vanilloid-1 (TRPV1) antagonists in human clinical trials: Insights from mathematical modeling and meta-analysis.

Antagonists of the transient receptor potential vanilloid-1 (TRPV1) channel alter body temperature (Tb) in laboratory animals and humans: most cause hyperthermia; some produce hypothermia; and yet others have no effect. TRPV1 can be activated by capsaicin (CAP), protons (low pH), and heat. First-generation (polymodal) TRPV1 antagonists potently block all three TRPV1 activation modes. Second-generation (mode-selective) TRPV1 antagonists potently block channel activation by CAP, but exert different effects (e.g., potentiation, no effect, or low-potency inhibition) in the proton mode, heat mode, or both. Based on our earlier studies in rats, only one mode of TRPV1 activation - by protons - is involved in thermoregulatory responses to TRPV1 antagonists. In rats, compounds that potently block, potentiate, or have no effect on proton activation cause hyperthermia, hypothermia, or no effect on Tb, respectively. A Tb response occurs when a TRPV1 antagonist blocks (in case of hyperthermia) or potentiates (hypothermia) the tonic TRPV1 activation by protons somewhere in the trunk, perhaps in muscles, and - via the acido-antithermogenic and acido-antivasoconstrictor reflexes - modulates thermogenesis and skin vasoconstriction. In this work, we used a mathematical model to analyze Tb data from human clinical trials of TRPV1 antagonists. The analysis suggests that, in humans, the hyperthermic effect depends on the antagonist's potency to block TRPV1 activation not only by protons, but also by heat, while the CAP activation mode is uninvolved. Whereas in rats TRPV1 drives thermoeffectors by mediating pH signals from the trunk, but not Tb signals, our analysis suggests that TRPV1 mediates both pH and thermal signals driving thermoregulation in humans. Hence, in humans (but not in rats), TRPV1 is likely to serve as a thermosensor of the thermoregulation system. We also conducted a meta-analysis of Tb data from human trials and found that polymodal TRPV1 antagonists (ABT-102, AZD1386, and V116517) increase Tb, whereas the mode-selective blocker NEO6860 does not. Several strategies of harnessing the thermoregulatory effects of TRPV1 antagonists in humans are discussed.

Trans-algorithmic nature of learning in biological systems.

Learning ability is a vitally important, distinctive property of biological systems, which provides dynamic stability in non-stationary environments. Although several different types of learning have been successfully modeled using a universal computer, in general, learning cannot be described by an algorithm. In other words, algorithmic approach to describing the functioning of biological systems is not sufficient for adequate grasping of what is life. Since biosystems are parts of the physical world, one might hope that adding some physical mechanisms and principles to the concept of algorithm could provide extra possibilities for describing learning in its full generality. However, a straightforward approach to that through the so-called physical hypercomputation so far has not been successful. Here an alternative approach is proposed. Biosystems are described as achieving enumeration of possible physical compositions though random incremental modifications inflicted on them by active operating resources (AORs) in the environment. Biosystems learn through algorithmic regulation of the intensity of the above modifications according to a specific optimality criterion. From the perspective of external observers, biosystems move in the space of different algorithms driven by random modifications imposed by the environmental AORs. A particular algorithm is only a snapshot of that motion, while the motion itself is essentially trans-algorithmic. In this conceptual framework, death of unfit members of a population, for example, is viewed as a trans-algorithmic modification made in the population as a biosystem by environmental AORs. Numerous examples of AOR utilization in biosystems of different complexity, from viruses to multicellular organisms, are provided.

Inclusion of neural effort in cost function can explain perceptual decision suboptimality.

A more general form of optimality approach applied to the entire behavioral paradigm should be used instead of abandoning the optimality approach. Adding the cost of information processing to the optimality criterion and taking into account some other recently proposed aspects of decision optimization could substantially increase the explanatory power of an optimality approach to modeling perceptual decision making.

Cold-Induced Thermogenesis and Inflammation-Associated Cold-Seeking Behavior Are Represented by Different Dorsomedial Hypothalamic Sites: A Three-Dimensional Functional Topography Study in Conscious Rats.

In the past, we showed that large electrolytic lesions of the dorsomedial hypothalamus (DMH) promoted hypothermia in cold-exposed restrained rats, but attenuated hypothermia in rats challenged with a high dose of bacterial lipopolysaccharide (LPS) in a thermogradient apparatus. The goal of this study was to identify the thermoeffector mechanisms and DMH representation of the two phenomena and thus to understand how the same lesion could produce two opposite effects on body temperature. We found that the permissive effect of large electrolytic DMH lesions on cold-induced hypothermia was due to suppressed thermogenesis. DMH-lesioned rats also could not develop fever autonomically: they did not increase thermogenesis in response to a low, pyrogenic dose of LPS (10 µg/kg, i.v.). In contrast, changes in thermogenesis were uninvolved in the attenuation of the hypothermic response to a high, shock-inducing dose of LPS (5000 µg/kg, i.v.); this attenuation was due to a blockade of cold-seeking behavior. To compile DMH maps for the autonomic cold defense and for the cold-seeking response to LPS, we studied rats with small thermal lesions in different parts of the DMH. Cold thermogenesis had the highest representation in the dorsal hypothalamic area. Cold seeking was represented by a site at the ventral border of the dorsomedial nucleus. Because LPS causes both fever and hypothermia, we originally thought that the DMH contained a single thermoregulatory site that worked as a fever-hypothermia switch. Instead, we have found two separate sites: one that drives thermogenesis and the other, previously unknown, that drives inflammation-associated cold seeking.SIGNIFICANCE STATEMENT Cold-seeking behavior is a life-saving response that occurs in severe systemic inflammation. We studied this behavior in rats with lesions in the dorsomedial hypothalamus (DMH) challenged with a shock-inducing dose of bacterial endotoxin. We built functional maps of the DMH and found the strongest representation of cold-seeking behavior at the ventral border of the dorsomedial nucleus. We also built maps for cold-induced thermogenesis in unanesthetized rats and found the dorsal hypothalamic area to be its main representation site. Our work identifies the neural substrate of cold-seeking behavior in systemic inflammation and expands the functional topography of the DMH, a structure that modulates autonomic, endocrine, and behavioral responses and is a potential therapeutic target in anxiety and panic disorders.

Two-phase strategy of neural control for planar reaching movements: II--relation to spatiotemporal characteristics of movement trajectory.

In the companion paper utilizing a quantitative model of optimal motor coordination (Part I, Rand and Shimansky, in Exp Brain Res 225:55-73, 2013), we examined coordination between X and Y movement directions (XYC) during reaching movements performed under three prescribed speeds, two movement amplitudes, and two target sizes. The obtained results indicated that the central nervous system (CNS) utilizes a two-phase strategy, where the initial and the final phases correspond to lower and higher precision of information processing, respectively, for controlling goal-directed reach-type movements to optimize the total cost of task performance including the cost of neural computations. The present study investigates how two different well-known concepts used for describing movement performance relate to the concepts of optimal XYC and two-phase control strategy. First, it is examined to what extent XYC is equivalent to movement trajectory straightness. The data analysis results show that the variability, the movement trajectory's deviation from the straight line, increases with an increase in prescribed movement speed. In contrast, the dependence of XYC strength on movement speed is opposite (in total agreement with an assumption of task performance optimality), suggesting that XYC is a feature of much higher level of generality than trajectory straightness. Second, it is tested how well the ballistic and the corrective components described in the traditional concept of two-component model of movement performance match with the initial and the final phase of the two-phase control strategy, respectively. In fast reaching movements, the percentage of trials with secondary corrective submovement was smaller under larger-target shorter-distance conditions. In slower reaching movements, meaningful parsing was impossible due to massive fluctuations in the kinematic profile throughout the movement. Thus, the parsing points determined by the conventional submovement analysis did not consistently reflect separation between the ballistic and error-corrective components. In contrast to the traditional concept of two-component movement performance, the concept of two-phase control strategy is applicable to a wide variety of experimental conditions.

Two-phase strategy of neural control for planar reaching movements: I. XY coordination variability and its relation to end-point variability.

A quantitative model of optimal transport-aperture coordination (TAC) during reach-to-grasp movements has been developed in our previous studies. The utilization of that model for data analysis allowed, for the first time, to examine the phase dependence of the precision demand specified by the CNS for neurocomputational information processing during an ongoing movement. It was shown that the CNS utilizes a two-phase strategy for movement control. That strategy consists of reducing the precision demand for neural computations during the initial phase, which decreases the cost of information processing at the expense of lower extent of control optimality. To successfully grasp the target object, the CNS increases precision demand during the final phase, resulting in higher extent of control optimality. In the present study, we generalized the model of optimal TAC to a model of optimal coordination between X and Y components of point-to-point planar movements (XYC). We investigated whether the CNS uses the two-phase control strategy for controlling those movements, and how the strategy parameters depend on the prescribed movement speed, movement amplitude and the size of the target area. The results indeed revealed a substantial similarity between the CNS's regulation of TAC and XYC. First, the variability of XYC within individual trials was minimal, meaning that execution noise during the movement was insignificant. Second, the inter-trial variability of XYC was considerable during the majority of the movement time, meaning that the precision demand for information processing was lowered, which is characteristic for the initial phase. That variability significantly decreased, indicating higher extent of control optimality, during the shorter final movement phase. The final phase was the longest (shortest) under the most (least) challenging combination of speed and accuracy requirements, fully consistent with the concept of the two-phase control strategy. This paper further discussed the relationship between motor variability and XYC variability.

Two-phase strategy of controlling motor coordination determined by task performance optimality.

A quantitative model of optimal coordination between hand transport and grip aperture has been derived in our previous studies of reach-to-grasp movements without utilizing explicit knowledge of the optimality criterion or motor plant dynamics. The model's utility for experimental data analysis has been demonstrated. Here we show how to generalize this model for a broad class of reaching-type, goal-directed movements. The model allows for measuring the variability of motor coordination and studying its dependence on movement phase. The experimentally found characteristics of that dependence imply that execution noise is low and does not affect motor coordination significantly. From those characteristics it is inferred that the cost of neural computations required for information acquisition and processing is included in the criterion of task performance optimality as a function of precision demand for state estimation and decision making. The precision demand is an additional optimized control variable that regulates the amount of neurocomputational resources activated dynamically. It is shown that an optimal control strategy in this case comprises two different phases. During the initial phase, the cost of neural computations is significantly reduced at the expense of reducing the demand for their precision, which results in speed-accuracy tradeoff violation and significant inter-trial variability of motor coordination. During the final phase, neural computations and thus motor coordination are considerably more precise to reduce the cost of errors in making a contact with the target object. The generality of the optimal coordination model and the two-phase control strategy is illustrated on several diverse examples.

Control of aperture closure initiation during trunk-assisted reach-to-grasp movements.

The present study investigated how the involvement and direction of trunk movement during reach-to-grasp movements affect the coordination between the transport and grasping components. Seated young adults made prehensile movements in which the involvement of the trunk was varied; the trunk was not involved, moved forward (flexion), or moved backward (extension) in the sagittal plane during the reach to the object. Each of the trunk movements was combined with an extension or flexion motion of the arm during the reach. Regarding the relationship between the trunk and arm motion for arm transport, the onset of wrist motion relative to that of the trunk was delayed to a greater extent for the trunk extension than for the trunk flexion. The variability of the time period from the peak of wrist velocity to the peak of trunk velocity was also significantly greater for trunk extension compared to trunk flexion. These findings indicate that trunk flexion was better integrated into the control of wrist transport than trunk extension. In terms of the temporal relationship between wrist transport and grip aperture, the relationship between the time of peak wrist velocity and the time of peak grip aperture did not change or become less steady across conditions. Therefore, the stability of temporal coordination between wrist transport and grip aperture was maintained despite the variation of the pattern of intersegmental coordination between the arm and the trunk during arm transport. The transport-aperture coordination was further assessed in terms of the control law according to which the initiation of aperture closure during the reach occurs when the hand crosses a hand-to-target distance threshold for grasp initiation, which is a function of peak aperture, wrist velocity and acceleration, trunk velocity and acceleration, and trunk-to-target distance at the time of aperture closure initiation. The participants increased the hand-to-target distance threshold for grasp initiation in the conditions where the trunk was involved compared to the conditions where the trunk was not involved. An increase also occurred when the trunk was extended compared to when it was flexed. The increased distance threshold implies an increase in the hand-to-target distance-related safety margin for grasping when the trunk is involved, especially when it is extended. These results suggest that the CNS significantly utilizes the parameters of trunk movement together with movement parameters related to the arm and the hand for controlling grasp initiation.

Adaptive force produced by stress-induced regulation of random variation intensity.

The Darwinian theory of life evolution is capable of explaining the majority of related phenomena. At the same time, the mechanisms of optimizing traits beneficial to a population as a whole but not directly to an individual remain largely unclear. There are also significant problems with explaining the phenomenon of punctuated equilibrium. From another perspective, multiple mechanisms for the regulation of the rate of genetic mutations according to the environmental stress have been discovered, but their precise functional role is not well understood yet. Here a novel mathematical paradigm called a Kinetic-Force Principle (KFP), which can serve as a general basis for biologically plausible optimization methods, is introduced and its rigorous derivation is provided. Based on this principle, it is shown that, if the rate of random changes in a biological system is proportional, even only roughly, to the amount of environmental stress, a virtual force is created, acting in the direction of stress relief. It is demonstrated that KFP can provide important insights into solving the above problems. Evidence is presented in support of a hypothesis that the nature employs KFP for accelerating adaptation in biological systems. A detailed comparison between KFP and the principle of variation and natural selection is presented and their complementarity is revealed. It is concluded that KFP is not a competing alternative, but a powerful addition to the principle of variation and natural selection. It is also shown KFP can be used in multiple ways for adaptation of individual biological organisms.

Contributions of different modes of TRPV1 activation to TRPV1 antagonist-induced hyperthermia.

Transient receptor potential vanilloid-1 (TRPV1) antagonists are widely viewed as next-generation pain therapeutics. However, these compounds cause hyperthermia, a serious side effect. TRPV1 antagonists differentially block three modes of TRPV1 activation: by heat, protons, and chemical ligands (e.g., capsaicin). We asked what combination of potencies in these three modes of TRPV1 activation corresponds to the lowest potency of a TRPV1 antagonist to cause hyperthermia. We studied hyperthermic responses of rats, mice, and guinea pigs to eight TRPV1 antagonists with different pharmacological profiles and used mathematical modeling to find a relative contribution of the blockade of each activation mode to the development of hyperthermia. We found that the hyperthermic effect has the highest sensitivity to the extent of TRPV1 blockade in the proton mode (0.43 to 0.65) with no to moderate sensitivity in the capsaicin mode (-0.01 to 0.34) and no sensitivity in the heat mode (0.00 to 0.01). We conclude that hyperthermia-free TRPV1 antagonists do not block TRPV1 activation by protons, even if they are potent blockers of the heat mode, and that decreasing the potency to block the capsaicin mode may further decrease the potency to cause hyperthermia.

Control of aperture closure initiation during reach-to-grasp movements under manipulations of visual feedback and trunk involvement in Parkinson's disease.

The present project was aimed at investigating how two distinct and important difficulties (coordination difficulty and pronounced dependency on visual feedback) in Parkinson's disease (PD) affect each other for the coordination between hand transport toward an object and the initiation of finger closure during reach-to-grasp movement. Subjects with PD and age-matched healthy subjects made reach-to-grasp movements to a dowel under conditions in which the target object and/or the hand were either visible or not visible. The involvement of the trunk in task performance was manipulated by positioning the target object within or beyond the participant's outstretched arm to evaluate the effects of increasing the complexity of intersegmental coordination under different conditions related to the availability of visual feedback in subjects with PD. General kinematic characteristics of the reach-to-grasp movements of the subjects with PD were altered substantially by the removal of target object visibility. Compared with the controls, the subjects with PD considerably lengthened transport time, especially during the aperture closure period, and decreased peak velocity of wrist and trunk movement without target object visibility. Most of these differences were accentuated when the trunk was involved. In contrast, these kinematic parameters did not change depending on the visibility of the hand for both groups. The transport-aperture coordination was assessed in terms of the control law according to which the initiation of aperture closure during the reach occurred when the hand distance-to-target crossed a hand-target distance threshold for grasp initiation that is a function of peak aperture, hand velocity and acceleration, trunk velocity and acceleration, and trunk-target distance at the time of aperture closure initiation. When the hand or the target object was not visible, both groups increased the hand-target distance threshold for grasp initiation compared to its value under full visibility, implying an increase in the hand-target distance-related safety margin for grasping. The increase in the safety margin due to the absence of target object vision or the absence of hand vision was accentuated in the subjects with PD compared to that in the controls. The pronounced increase in the safety margin due to absence of target object vision for the subjects with PD was further accentuated when the trunk was involved compared to when it was not involved. The results imply that individuals with PD have significant limitations regarding neural computations required for efficient utilization of internal representations of target object location and hand motion as well as proprioceptive information about the hand to compensate for the lack of visual information during the performance of complex multisegment movements.

Biologically plausible learning in neural networks: a lesson from bacterial chemotaxis.

Learning processes in the brain are usually associated with plastic changes made to optimize the strength of connections between neurons. Although many details related to biophysical mechanisms of synaptic plasticity have been discovered, it is unclear how the concurrent performance of adaptive modifications in a huge number of spatial locations is organized to minimize a given objective function. Since direct experimental observation of even a relatively small subset of such changes is not feasible, computational modeling is an indispensable investigation tool for solving this problem. However, the conventional method of error back-propagation (EBP) employed for optimizing synaptic weights in artificial neural networks is not biologically plausible. This study based on computational experiments demonstrated that such optimization can be performed rather efficiently using the same general method that bacteria employ for moving closer to an attractant or away from a repellent. With regard to neural network optimization, this method consists of regulating the probability of an abrupt change in the direction of synaptic weight modification according to the temporal gradient of the objective function. Neural networks utilizing this method (regulation of modification probability, RMP) can be viewed as analogous to swimming in the multidimensional space of their parameters in the flow of biochemical agents carrying information about the optimality criterion. The efficiency of RMP is comparable to that of EBP, while RMP has several important advantages. Since the biological plausibility of RMP is beyond a reasonable doubt, the RMP concept provides a constructive framework for the experimental analysis of learning in natural neural networks.

Role of vision in aperture closure control during reach-to-grasp movements.

We have previously shown that the distance from the hand to the target at which finger closure is initiated during the reach (aperture closure distance) depends on the amplitude of peak aperture, as well as hand velocity and acceleration. This dependence suggests the existence of a control law according to which a decision to initiate finger closure during the reach is made when the hand distance to target crosses a threshold that is a function of the above movement-related parameters. The present study examined whether the control law is affected by manipulating the visibility of the hand and the target. Young adults made reach-to-grasp movements to a dowel under conditions in which the target or the hand or both were either visible or not visible. Reaching for and grasping a target when the hand and/or target were not visible significantly increased transport time and widened peak aperture. Aperture closure distance was significantly lengthened and wrist peak velocity was decreased only when the target was not visible. Further analysis showed that the control law was significantly different between the visibility-related conditions. When either the hand or target was not visible, the aperture closure distance systematically increased compared to its value for the same amplitude of peak aperture, hand velocity, and acceleration under full visibility. This implies an increase in the distance-related safety margin for grasping when the hand or target is not visible. It has been also found that the same control law can be applied to all conditions, if variables describing hand and target visibility were included in the control law model, as the parameters of the task-related environmental context, in addition to the above movement-related parameters. This suggests that that the CNS utilizes those variables for controlling grasp initiation based on a general control law.

A novel model of motor learning capable of developing an optimal movement control law online from scratch.

A computational model of a learning system (LS) is described that acquires knowledge and skill necessary for optimal control of a multisegmental limb dynamics (controlled object or CO), starting from "knowing" only the dimensionality of the object's state space. It is based on an optimal control problem setup different from that of reinforcement learning. The LS solves the optimal control problem online while practicing the manipulation of CO. The system's functional architecture comprises several adaptive components, each of which incorporates a number of mapping functions approximated based on artificial neural nets. Besides the internal model of the CO's dynamics and adaptive controller that computes the control law, the LS includes a new type of internal model, the minimal cost (IM(mc)) of moving the controlled object between a pair of states. That internal model appears critical for the LS's capacity to develop an optimal movement trajectory. The IM(mc) interacts with the adaptive controller in a cooperative manner. The controller provides an initial approximation of an optimal control action, which is further optimized in real time based on the IM(mc). The IM(mc) in turn provides information for updating the controller. The LS's performance was tested on the task of center-out reaching to eight randomly selected targets with a 2DOF limb model. The LS reached an optimal level of performance in a few tens of trials. It also quickly adapted to movement perturbations produced by two different types of external force field. The results suggest that the proposed design of a self-optimized control system can serve as a basis for the modeling of motor learning that includes the formation and adaptive modification of the plan of a goal-directed movement.

Selected contribution: ambient temperature for experiments in rats: a new method for determining the zone of thermal neutrality.

There is a misbelief that the same animal has the same thermoneutral zone (TNZ) in different experimental setups. In reality, TNZ strongly depends on the physical environment and varies widely across setups. Current methods for determining TNZ require elaborate equipment and can be applied only to a limited set of experimental conditions. A new, broadly applicable approach that rapidly determines whether given conditions are neutral for a given animal is needed. Consistent with the definition of TNZ [the range of ambient temperature (T(a)) at which body core temperature (T(c)) regulation is achieved only by control of sensible heat loss], we propose three criteria of thermoneutrality: 1) the presence of high-magnitude fluctuations in skin temperature (T(sk)) of body parts serving as specialized heat exchangers with the environment (e.g., rat tail), 2) the closeness of T(sk) to the median of its operational range, and 3) a strong negative correlation between T(sk) and T(c). Thermocouple thermometry and liquid crystal thermography were performed in five rat strains at 13 T(a). Under the conditions tested (no bedding or filter tops, no group thermoregulation), the T(a) range of 29.5-30.5 degrees C satisfied all three TNZ criteria in Wistar, BDIX, Long-Evans, and Zucker lean rats; Zucker fatty rats had a slightly lower TNZ (28.0-29.0 degrees C). Skin thermometry or thermography is a definition-based, simple, and inexpensive technique to determine whether experimental or housing conditions are neutral, subneutral, or supraneutral for a given animal.