Diverse Imitation Learning via Self-Organizing Generative Models.

Imitation learning (IL) is a well-known problem in the field of Markov decision process (MDP), where one is given multiple demonstration trajectories generated by expert(s), and the goal is to replicate the hidden expert-policies so that when the MDP is run independently, it generates trajectories close to the demonstrated ones. IL is one of the most useful tools used in building versatile robots that can learn from examples. This task becomes particularly challenging when the expert exhibits a mixture of behavior modes. Prior work has introduced latent variables to model variations of the expert policy. However, our experiments show that the existing works do not exhibit appropriate imitation of individual modes. To tackle this problem, we first draw inspiration from the well-known classical technique of self-organizing maps (SOMs) and introduce an encoder-free generative model-referred to as the self-organizing generative (SOG) model-for learning multimodal data distributions from samples. We then apply SOG for behavior cloning (BC)-a framework that learns deterministic policies without considering the environment-to accurately distinguish and imitate different modes. Then, we integrate it with generative adversarial IL (GAIL)-a framework that learns policies while considering the environment-to make the learning robust toward compounding errors at unseen states. We show that our method significantly outperforms the state of the art across multiple experiments within the MuJoCo simulator, including locomotion and robotic manipulation tasks.

Optimizing detection and deep learning-based classification of pathological high-frequency oscillations in epilepsy.

OBJECTIVE: This study aimed to explore sensitive detection methods for pathological high-frequency oscillations (HFOs) to improve seizure outcomes in epilepsy surgery. METHODS: We analyzed interictal HFOs (80-500 Hz) in 15 children with medication-resistant focal epilepsy who underwent chronic intracranial electroencephalogram via subdural grids. The HFOs were assessed using the short-term energy (STE) and Montreal Neurological Institute (MNI) detectors and examined for spike association and time-frequency plot characteristics. A deep learning (DL)-based classification was applied to purify pathological HFOs. Postoperative seizure outcomes were correlated with HFO-resection ratios to determine the optimal HFO detection method. RESULTS: The MNI detector identified a higher percentage of pathological HFOs than the STE detector, but some pathological HFOs were detected only by the STE detector. HFOs detected by both detectors had the highest spike association rate. The Union detector, which detects HFOs identified by either the MNI or STE detector, outperformed other detectors in predicting postoperative seizure outcomes using HFO-resection ratios before and after DL-based purification. CONCLUSIONS: HFOs detected by standard automated detectors displayed different signal and morphological characteristics. DL-based classification effectively purified pathological HFOs. SIGNIFICANCE: Enhancing the detection and classification methods of HFOs will improve their utility in predicting postoperative seizure outcomes.

Optimizing Detection and Deep Learning-based Classification of Pathological High-Frequency Oscillations in Epilepsy.

Objective: This study aimed to explore sensitive detection methods and deep learning (DL)-based classification for pathological high-frequency oscillations (HFOs). Methods: We analyzed interictal HFOs (80-500 Hz) in 15 children with medication-resistant focal epilepsy who underwent resection after chronic intracranial electroencephalogram via subdural grids. The HFOs were assessed using the short-term energy (STE) and Montreal Neurological Institute (MNI) detectors and examined for pathological features based on spike association and time-frequency plot characteristics. A DL-based classification was applied to purify pathological HFOs. Postoperative seizure outcomes were correlated with HFO-resection ratios to determine the optimal HFO detection method. Results: The MNI detector identified a higher percentage of pathological HFOs than the STE detector, but some pathological HFOs were detected only by the STE detector. HFOs detected by both detectors exhibited the most pathological features. The Union detector, which detects HFOs identified by either the MNI or STE detector, outperformed other detectors in predicting postoperative seizure outcomes using HFO-resection ratios before and after DL-based purification. Conclusions: HFOs detected by standard automated detectors displayed different signal and morphological characteristics. DL-based classification effectively purified pathological HFOs. Significance: Enhancing the detection and classification methods of HFOs will improve their utility in predicting postoperative seizure outcomes. HIGHLIGHTS: HFOs detected by the MNI detector showed different traits and higher pathological bias than those detected by the STE detectorHFOs detected by both MNI and STE detectors (the Intersection HFOs) were deemed the most pathologicalA deep learning-based classification was able to distill pathological HFOs, regard-less of the initial HFO detection methods.

Decoding of human identity by computer vision and neuronal vision.

Extracting meaning from a dynamic and variable flow of incoming information is a major goal of both natural and artificial intelligence. Computer vision (CV) guided by deep learning (DL) has made significant strides in recognizing a specific identity despite highly variable attributes. This is the same challenge faced by the nervous system and partially addressed by the concept cells-neurons exhibiting selective firing in response to specific persons/places, described in the human medial temporal lobe (MTL) ⁠. Yet, access to neurons representing a particular concept is limited due to these neurons' sparse coding. It is conceivable, however, that the information required for such decoding is present in relatively small neuronal populations. To evaluate how well neuronal populations encode identity information in natural settings, we recorded neuronal activity from multiple brain regions of nine neurosurgical epilepsy patients implanted with depth electrodes, while the subjects watched an episode of the TV series "24". First, we devised a minimally supervised CV algorithm (with comparable performance against manually-labeled data) to detect the most prevalent characters (above 1% overall appearance) in each frame. Next, we implemented DL models that used the time-varying population neural data as inputs and decoded the visual presence of the four main characters throughout the episode. This methodology allowed us to compare "computer vision" with "neuronal vision"-footprints associated with each character present in the activity of a subset of neurons-and identify the brain regions that contributed to this decoding process. We then tested the DL models during a recognition memory task following movie viewing where subjects were asked to recognize clip segments from the presented episode. DL model activations were not only modulated by the presence of the corresponding characters but also by participants' subjective memory of whether they had seen the clip segment, and by the associative strengths of the characters in the narrative plot. The described approach can offer novel ways to probe the representation of concepts in time-evolving dynamic behavioral tasks. Further, the results suggest that the information required to robustly decode concepts is present in the population activity of only tens of neurons even in brain regions beyond MTL.

Differential effects of delayed cord clamping on bilirubin levels in normal and diabetic pregnancies.

The purpose of the study is to investigate the effects of delayed cord clamping on bilirubin levels and phototherapy rates in neonates of diabetic mothers. This was a prospective study that enrolled pregnant women without pregnancy complications and those with diabetes. Their neonates were randomized in a 1:1 ratio to delayed cord clamping. The main outcomes were the neonatal transcutaneous bilirubin values on 2-4 days postpartum and the rate of requiring phototherapy in infants. A total of 261 pregnant women were included in the final analysis (132 women with diabetic pregnancies and 129 women with normal pregnancies). In diabetic pregnancies, neonatal bilirubin levels on the 2-4 days postpartum and phototherapy rates were significantly higher in the delayed cord clamping group than in the immediate cord clamping group (7.65 ± 1.83 vs 8.25 ± 1.96, P = 0.039; 10.35 ± 2.23 vs 11.54 ± 2.56, P = 0.002; 11.54 ± 2.94 vs 12.83 ± 3.07 P = 0.024, 18.2% vs 6.3%, P = 0.042), while in normal pregnancies, there was no statistical difference in bilirubin values and phototherapy rates between the delayed cord clamping group and the immediate cord clamping group (P > 0.05). After receiving delayed cord clamping, bilirubin levels on the third postnatal day and the rate of requiring phototherapy in infants were higher in the diabetic pregnancy group than in the normal pregnancy group (10.35 ± 2.23 vs 11.54 ± 2.56, P = 0.013). CONCLUSION: Delayed cord clamping increased the risk of jaundice in newborns born to diabetic mothers, but had no effect in newborns from mothers with normal pregnancies. DCC may be a risk factor for increased bilirubin in infants of diabetic mothers. TRIAL REGISTRATION: ClinicalTrials.gov: NCT04369313; date of registration: April 27, 2020 (retrospectively registered). WHAT IS KNOWN: • Delayed cord clamping had significant benefits for newborns by increasing neonatal hemoglobin levels and reducing the risk of neonatal anemia, etc. • Delayed cord clamping may lead to neonatal hyperemia, erythrocytosis, and hyperbilirubinemia, which increases the risk of neonatal jaundice. WHAT IS NEW: • Our trial focused on the differential effects of delayed cord clamping on jaundice in full-term newborns between diabetic pregnancies and normal pregnancies. And newborns of diabetic mothers who received delayed cord clamping had a significantly increased risk of jaundice compared to newborns with normal pregnancy. • Delayed cord clamping may be a risk factor for increased bilirubin levels in neonates of diabetic mothers.

Refining epileptogenic high-frequency oscillations using deep learning: a reverse engineering approach.

Intracranially recorded interictal high-frequency oscillations have been proposed as a promising spatial biomarker of the epileptogenic zone. However, its visual verification is time-consuming and exhibits poor inter-rater reliability. Furthermore, no method is currently available to distinguish high-frequency oscillations generated from the epileptogenic zone (epileptogenic high-frequency oscillations) from those generated from other areas (non-epileptogenic high-frequency oscillations). To address these issues, we constructed a deep learning-based algorithm using chronic intracranial EEG data via subdural grids from 19 children with medication-resistant neocortical epilepsy to: (i) replicate human expert annotation of artefacts and high-frequency oscillations with or without spikes, and (ii) discover epileptogenic high-frequency oscillations by designing a novel weakly supervised model. The 'purification power' of deep learning is then used to automatically relabel the high-frequency oscillations to distill epileptogenic high-frequency oscillations. Using 12 958 annotated high-frequency oscillation events from 19 patients, the model achieved 96.3% accuracy on artefact detection (F1 score = 96.8%) and 86.5% accuracy on classifying high-frequency oscillations with or without spikes (F1 score = 80.8%) using patient-wise cross-validation. Based on the algorithm trained from 84 602 high-frequency oscillation events from nine patients who achieved seizure-freedom after resection, the majority of such discovered epileptogenic high-frequency oscillations were found to be ones with spikes (78.6%, P < 0.001). While the resection ratio of detected high-frequency oscillations (number of resected events/number of detected events) did not correlate significantly with post-operative seizure freedom (the area under the curve = 0.76, P = 0.06), the resection ratio of epileptogenic high-frequency oscillations positively correlated with post-operative seizure freedom (the area under the curve = 0.87, P = 0.01). We discovered that epileptogenic high-frequency oscillations had a higher signal intensity associated with ripple (80-250 Hz) and fast ripple (250-500 Hz) bands at the high-frequency oscillation onset and with a lower frequency band throughout the event time window (the inverted T-shaped), compared to non-epileptogenic high-frequency oscillations. We then designed perturbations on the input of the trained model for non-epileptogenic high-frequency oscillations to determine the model's decision-making logic. The model confidence significantly increased towards epileptogenic high-frequency oscillations by the artificial introduction of the inverted T-shaped signal template (mean probability increase: 0.285, P < 0.001), and by the artificial insertion of spike-like signals into the time domain (mean probability increase: 0.452, P < 0.001). With this deep learning-based framework, we reliably replicated high-frequency oscillation classification tasks by human experts. Using a reverse engineering technique, we distinguished epileptogenic high-frequency oscillations from others and identified its salient features that aligned with current knowledge.

Effects of delayed cord clamping on neonatal jaundice, phototherapy and early hematological status in term cesarean section.

BACKGROUND: Delayed cord clamping in full-term neonates is widely recommended, while in practice, it is rarely implemented in cesarean section due to the fear of neonatal jaundice and excessive maternal blood loss. The optimal timing of cord clamping remains uncertain. This study was to fully evaluate the effects of delayed cord clamping on short-term hematological status and jaundice in term neonates delivered by cesarean section. METHODS: This retrospective study enrolled 796 women, who were allocated into the early cord clamping group (n = 377) and the delayed cord clamping group (n = 419). The latter group was further divided into two subgroups (30-60 s, 61-120 s). The outcomes were neonatal transcutaneous bilirubin levels on 0 to 5 days of life and the rate of phototherapy. For neonates who had blood tests on the first three days of life, their hemoglobin and hematocrit were compared among groups. RESULTS: Compared with the early cord clamping group, delayed cord clamping merely increased the transcutaneous bilirubin level of neonates on the day of birth rather than that on the following five days. The heel peripheral blood sample size of 1-3 days in the early cord clamping group was 61, 25 and 33, and in the delayed cord clamping group was 53, 46 and 32, respectively. Delayed cord clamping at 30-60 s resulted in the higher neonatal hemoglobin level on day 3 and an increased rate of neonatal polycythemia, without a higher rate of phototherapy. Delayed cord clamping beyond 60 s did not further improve hematological status in term neonates born by cesarean section. CONCLUSION: In cesarean section, delayed cord clamping for 30-60 s improved the early hematological status of term neonates without the enhanced requirement of phototherapy for neonatal jaundice.

Timing of umbilical cord clamping and neonatal jaundice in singleton term pregnancy.

BACKGROUND: Delayed cord clamping was not adopted widely in China because of the potential effect of neonatal hyperbilirubinemia, jaundice and polycythemia, and the optimal cord clamping time remained controversial. AIM: To assess the effect of delayed cord clamping versus early cord clamping on neonatal jaundice for term infants. STUDY DESIGN: This retrospective study included 1981 mother-infant pairs, who were assigned to early cord clamping groups (n = 1005) and delayed cord clamping group (n = 949). The delayed cord clamping included three subgroups (30-60 s, 61-90 s, 91-120 s). The main outcomes were transcutaneous bilirubin levels at 0 to 4 days of age, the rate of jaundice requiring phototherapy, the neonatal hematological status at 1 to 3 days after birth. RESULTS: Compared with the early cord clamping group, the neonatal transcutaneous bilirubin level did not differ and the neonatal hematological status (hemoglobin and hematocrit levels) were improved in combined and three subgroups of delayed cord clamping group. Increasing the duration of cord clamping from 90 s to 120 s did not result in further increases in hemoglobin and hematocrit levels but led to a trend towards a higher risk of neonatal jaundice requiring phototherapy and neonatal polycythemia. CONCLUSIONS: Delayed cord clamping for <90 s in healthy term infants may not only improve the early hematological status of newborns but also avoid excessive neonatal jaundice requiring phototherapy.