Control of chaotic systems by deep reinforcement learning.

Deep reinforcement learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for the determination of optimal control solutions and deep neural networks for approximating the value function and the control policy. Recent applications have shown that DRL may achieve superhuman performance in complex cognitive tasks. In this work, we show that using restricted localized actuation, partial knowledge of the state based on limited sensor measurements and model-free DRL controllers, it is possible to stabilize the dynamics of the KS system around its unstable fixed solutions, here considered as target states. The robustness of the controllers is tested by considering several trajectories in the phase space emanating from different initial conditions; we show that DRL is always capable of driving and stabilizing the dynamics around target states. The possibility of controlling the KS system in the chaotic regime by using a DRL strategy solely relying on local measurements suggests the extension of the application of RL methods to the control of more complex systems such as drag reduction in bluff-body wakes or the enhancement/diminution of turbulent mixing.

Human biodistribution and dosimetry of iodine-123-fluoroalkyl analogs of beta-CIT.

Two new N-omega-fluoroalkyl analogs of [123I]2beta-carbomethoxy-3beta-(4-iodophenyl)tropane ([123I]beta-CIT), the fluoroethyl and fluoropropyl compounds ([123I]FE-CIT and [123I]FP-CIT, respectively), have been shown to have faster kinetics and better selectivity for the dopamine transporter than [123I]beta-CIT. We examined the organ biodistribution and radiation safety of these two compounds in six healthy volunteers who received an injection with each of the two compounds 2 weeks apart. Data were obtained on the Strichman 860 whole-body scanner. Transmission scans were obtained in all subjects prior to the injection of the radiotracer with a line source and used to derive organ-specific attenuation correction factors. Whole-body planar images were acquired every hour for the first 6 h, and at 24 h. Attenuation-corrected regional conjugate counts were converted into units of activity using a calibration factor obtained for each subject by dividing whole-body conjugate decay-corrected counts from the first acquisition by the injected activity. Radiation dose estimates were on average higher for [123I]CIT-FE than for [123I]CIT-FP, with the lower large intestine receiving the highest exposure: 0.15+/-13% mGy/MBq (mean +/-COV) and 0.12+/-14% mGy/MBq for [123I]FE-CIT and [123I]FP-CIT, respectively, followed by the upper large intestine and the spleen.

Comparison of technetium-99m-HMPAO and technetium-99m-ECD cerebral SPECT images in Alzheimer's disease.

UNLABELLED: SPECT has shown increasing promise as a diagnostic tool in Alzheimer's disease (AD). Recently, a new SPECT brain perfusion agent, 99mTc-ethyl cysteinate dimer (99mTc-ECD) has emerged with purported advantages in image quality over the established tracer, 99mTc-hexamethylpropyleneamine oxime (99mTc-HMPAO). This research aimed to compare cerebral images for 99mTc-HMPAO and 99mTc-ECD in discriminating patients with AD from control subjects. METHODS: Twenty-four AD patients (mean age +/- s.d. = 68.9 +/- 8.2 yr) and 13 healthy subjects (68.4 +/- 8.0 yr) were scanned sequentially with 20 mCi of each tracer using the CERASPECT system within 1 mo. Scanning began on average 11.5 +/- 2.8 min after 99mTc-HMPAO injection and 41.8 +/- 10.1 min after 99mTc-ECD. A ratio, R, was derived of count densities in "typically affected" brain structures (parietal and temporal association cortices) to "unaffected" structures (cerebellum, basal ganglia, thalamus, occipital cortex, and sensorimotor cortex). RESULTS: Analysis of variance revealed significant interaction between diagnostic group and radiopharmaceutical (F = 4.71; df = 1.35; p = 0.04), with 99mTc-ECD demonstrating better separation of R values between AD patients and control subjects than 99mTc-HMPAO. Receiver operating characteristic (ROC) analysis, revealed no significant difference in the ability of the two tracers to correctly classify AD patients and control subjects. Both tracers showed high diagnostic accuracy (99mTc-ECD: sensitivity = 100%, specificity = 92%; 99mTc-HMPAO: sensitivity = 100%, specificity = 85%). CONCLUSION: Technetium-99m-ECD shows greater contrast than 99mTc-HMPAO between affected and unaffected brain structures in AD when patients are compared to age-matched control subjects. Both tracers perform equally well in correctly classifying patients and control subjects.

Acute cocaine effects on absolute cerebral blood flow.

Cocaine use has been associated with vasoconstriction and stroke, and several studies have demonstrated that it decreases relative cerebral blood flow (rCBF) in humans. However, rCBF has not been quantitated. We compared 40 mg IV cocaine hydrochloride to placebo effects on absolute rCBF in four cocaine users using 99mTc-HMPAO SPECT with a modified microsphere model for CBF quantitation. Cocaine produced significant decreases in rCBF in all regions studied with a mean decrease of 30% in absolute whole brain blood flow (P = 0.002) which was 3-fold greater than relative blood flow changes.

Human biodistribution and dosimetry of the SPECT D2 dopamine receptor radioligand [123I]IBF.

The research discussed in this article aimed to characterize better the biodistribution, excretion and radiation dosimetry of the single photon emission computed tomography (SPECT) D2 Dopamine receptor radioligand [123I]IBF. Following administration of 111 +/- 12 MBq [123I]IBF, seven healthy human subjects were scanned serially with a whole body imager over a 48-h period. Transmission images were obtained with a scanning line source for attenuation correction of the emission images. Urine was collected for 48 h to measure the fraction of activity voided by the renal system. Radiation absorbed dose estimates were performed using biokinetic modeling and the Medical Internal Radiation Dose (MIRD) schema. Highest absorbed doses were to the kidney (0.13 +/- 0.02 mGy/MBq) and urinary bladder wall (0.11 +/- 0.01 mGy/MBq). The effective dose equivalent was 0.041 +/- 0.005 mSv/MBq. Peak brain uptake represented 8% of the injected activity. Rapid urinary excretion minimized the absorbed dose to most tissues. The mean cumulative urinary excretion fraction was 69%. Thus [123I]IBF is a promising SPECT agent for imaging the D2 dopamine receptor in humans with high brain uptake and favorable dosimetry.

Difference images calculated from ictal and interictal technetium-99m-HMPAO SPECT scans of epilepsy.

UNLABELLED: Image processing techniques were applied to SPECT brain images to aid in the localization of epileptic foci. METHODS: Ictal and interictal cerebral perfusion SPECT images were acquired from 12 epilepsy patients (6 temporal, 6 extratemporal) after injection of 20 mCi 99mTc-HMPAO. Each ictal scan was registered to the same patient's interictal scan. Normalization of the three-dimensional data was applied to account for global percent brain uptake and total injected activity. After registration, normalization and subtraction of the SPECT images and functional difference images were computed. Difference images were calculated, which give a quantitative measure of perfusion alterations during ictus. The resulting difference images were also registered with each patient's MRI scan which permits localization of perfusion changes onto anatomical structures. RESULTS: Areas in the brain where significant perfusion increases occur correlate with areas confirmed to be seizure foci. Four of the six patients with known temporal lobe seizure foci exhibited significant perfusion increases on the difference images. These areas demonstrate a percent increase of perfusion larger than 40%. For the extratemporal seizure patients, four of the four confirmed seizure sites were diagnosed with difference images. Results on the remaining two patients were inconclusive. CONCLUSION: When compared to side-by-side visual interpretation of the ictal and interictal SPECT images, registration of SPECT and MR images together with calculated difference maps greatly enhances the ability to localize epileptic seizure foci. This offers the potential to locate epileptic seizure foci using a noninvasive, inexpensive imaging procedure and data processing algorithm.

SPECT imaging of dopamine and serotonin transporters with [123I]beta-CIT: pharmacological characterization of brain uptake in nonhuman primates.

Single photon emission computed tomography (SPECT) studies of regional kinetic uptake and pharmacological specificity of [123I]methyl 3 beta-(4-iodophenyl) tropane-2 beta-carboxylate ([123I]beta-CIT) were performed in nonhuman primates (n = 41). In control experiments, activity was concentrated in striatum and in hypothalamic/midbrain regions. Striatal uptake increased for 140-180 min and displayed stable levels thereafter. Striatal to cerebellar activity ratios were 7.3 +/- 0.9 (mean +/- SEM) at 300 min. About 75% of striatal uptake was displaceable by injection of nonradioactive beta-CIT. Hypothalamic/midbrain activity reached maximal levels at approximately 45 min. A slow washout phase followed this peak activity. Activities in frontal, occipital, and cerebellar regions were characterized by an early peak (20-30 min), followed by rapid washout. Displacement studies demonstrated that striatal uptake was associated with dopamine (DA) transporters, as it was displaced by GBR 12909, a selective DA uptake inhibitor, but not by citalopram, a selective serotonin (5-HT) uptake inhibitor. The inverse was true in the hypothalamic/midbrain area, suggesting that the uptake in this area was associated primarily with 5-HT transporters. Maprotiline, a selective norepinephrine uptake inhibitor, did not affect [123I]beta-CIT uptake. In vivo site occupancy ED50 values of cocaine, 2 beta-carbomethoxy-3 beta-(4-fluorophenyl)tropane (CFT), and beta-CIT were measured in the striatum with a stepwise displacement paradigm. In vivo ED50 values correlated strongly with in vitro IC50 values for binding to DA transporters. Infusion of high dose of L-DOPA (250 mumol/kg) failed to displace striatal [123I]beta-CIT binding, suggesting that the binding would not be affected by L-DOPA administration in Parkinsonian patients. However, studies performed with injection of d-amphetamine indirectly suggested that high synaptic levels of DA may compete with [123I]beta-CIT binding. These studies suggest that [123I]beta-CIT will be a useful SPECT tracer of DA and 5-HT transporters in living human brain.

Different gamma-aminobutyric acid receptor subtypes are involved in the regulation of opiate-dependent and independent luteinizing hormone-releasing hormone secretion.

The neurotransmitter gamma-aminobutyric acid (GABA) appears to be involved in the control of gonadotropin secretion. These studies were conducted 1) to evaluate the effect of GABAergic drugs on in vitro LHRH secretion and 2) to characterize the role of different types of GABA receptors (the GABA-A and GABA-B subtypes) in these actions. Arcuate nuclei-median eminence fragments were incubated in vitro, and the release of LHRH, prostaglandin E2 (PGE2), arginine vasopressin, and oxytocin was measured by RIA. Both GABA and muscimol at different concentrations induced an increase in LHRH release, but did not affect the release of arginine vasopressin and oxytocin. This stimulatory effect was blocked by the specific GABA antagonist bicuculline, suggesting the involvement of GABA-A type receptors. Muscimol-stimulated LHRH release was not affected by the presence of phentolamine, suggesting that the stimulatory effect of GABA-A receptors on LHRH release is not mediated by interactions with the noradrenergic system. PGE2 has been shown to be a potent secretagogue of LHRH from the median eminence in vitro, and in this model the stimulatory effect of PGE2 was enhanced by muscimol. Baclofen, a specific GABA-B type receptor agonist, had no effect on basal LHRH release, but completely suppressed naloxone-stimulated LHRH and PGE2 secretion. The inhibitory effect of baclofen was blocked by the presence of 5-aminovalerate, a drug that has been shown to block the inhibitory effect of baclofen on NE release from noradrenergic terminals. This suggests the possibility that GABA-B receptors interacting with noradrenergic terminals may be responsible for the inhibitory effect of baclofen on naloxone stimulation. This study uncovered both stimulatory and inhibitory effects of GABA on LHRH release after activation of GABA-A or GABA-B receptors, respectively. Further, the data show possible relationships among the GABAergic, endogenous opiate peptide, and noradrenergic systems in the control of LHRH release from the hypothalamus.

Pulsatile peptide secretion: encoding of brain messages regulating endocrine and reproductive functions.

Neuropeptides are defined chemical messengers produced by the brain to modulate its own activity and also to regulate the function of every organ system. These neuropeptides can be viewed as coded chemical signals produced by the brain and secreted into the blood or into other fluids, such as the cerebrospinal fluid, to be transported and to act at a distant site. The signals arrive to the target organ or sometimes to an intermediary station, such as the pituitary gland, where they are decoded, transformed into a more powerful signal, and sent again through the general circulation to reach their final target. Our work has characterized the episodic or pulsatile pattern of secretion of a number of peptide hormones produced by the brain or the pituitary gland and analyzed the brain mechanisms involved in the generation of such a pulsatile pattern of hormone secretion. Molecular biology approaches have provided information on the synthesis, processing, and secretion of these brain messengers. In addition, using computer-assisted perifusion systems, we have been able to reproduce in vitro some of the signals produced by the brain and are currently trying to decode the message carried by those signals, as well as determining the intracellular messengers involved in the signal process. The importance of the neuropeptides and of the messages carried by the pulsatile signal is underlined by experiments in which animals treated with a neurotoxin were rendered infertile.(ABSTRACT TRUNCATED AT 250 WORDS)