Compact Goal Representation Learning via Information Bottleneck in Goal-Conditioned Reinforcement Learning.

We propose an Information bottleneck (IB) for Goal representation learning (InfoGoal), a self-supervised method for generalizable goal-conditioned reinforcement learning (RL). Goal-conditioned RL learns a policy from reward signals to predict actions for reaching desired goals. However, the policy would overfit the task-irrelevant information contained in the goal and may be falsely or ineffectively generalized to reach other goals. A goal representation containing sufficient task-relevant information and minimum task-irrelevant information is guaranteed to reduce generalization errors. However, in goal-conditioned RL, it is difficult to balance the tradeoff between task-relevant information and task-irrelevant information because of the sparse and delayed learning signals, i.e., reward signals, and the inevitable task-relevant information sacrifice caused by information compression. Our InfoGoal learns a minimum and sufficient goal representation with dense and immediate self-supervised learning signals. Meanwhile, InfoGoal adaptively adjusts the weight of information minimization to achieve maximum information compression with a reasonable sacrifice of task-relevant information. Consequently, InfoGoal enables policy to generate a targeted trajectory toward states where the desired goal can be found with high probability and broadly explores those states. We conduct experiments on both simulated and real-world tasks, and our method significantly outperforms baseline methods in terms of policy optimality and the success rate of reaching unseen test goals. Video demos are available at infogoal.github.io.

[Construction and validation of a decision tree based on biomarkers for predicting severe acute kidney injury in critically ill patients].

OBJECTIVE: To construct and evaluate a decision tree based on biomarkers for predicting severe acute kidney injury (AKI) in critical patients. METHODS: A prospectively study was conducted. Critical patients who had been admitted to the department of critical care medicine of Xiaolan Hospital of Southern Medical University from January 2017 to June 2018 were enrolled. The clinical data of the patients were recorded, and the biomarkers, including serum cystatin C (sCys C) and urinary N-acetyl-ß-D-glucosaminidase (uNAG) were established immediately after admission to intensive care unit (ICU), and the end points were recorded. The test cohort was established with patient data from January to December 2017. The decision tree classification and regression tree (CART) algorithm was used, and the best cut-off values of biomarkers were used as the decision node to construct a biomarker decision tree model for predicting severe AKI. The accuracy of the decision tree model was evaluated by the overall accuracy and the receiver operating characteristic (ROC) curve. The validation cohort, established on patient data from January to June 2018, was used to further validate the accuracy and predictive ability of the decision tree. RESULTS: In test cohort, 263 patients were enrolled, of whom 57 developed severe AKI [defined as phase 2 and 3 of Kidney Disease: Improving Global Outcomes (KDIGO) criterion]. Compared with patients without severe AKI, severe AKI patients were older [years old: 64 (49, 74) vs. 52 (41, 66)], acute physiology and chronic health evaluation II (APACHE II) score were higher [23 (19, 27) vs. 15 (11, 20)], the incidence of hypertension, diabetes and other basic diseases and sepsis were higher (64.9% vs. 40.3%, 28.1% vs. 10.7%, 63.2% vs. 29.6%), the levels of sCys C and uNAG were higher [sCys C (mg/L): 1.38 (1.12, 2.02) vs. 0.79 (0.67, 0.98), uNAG (U/mmol Cr): 5.91 (2.43, 10.68) vs. 2.72 (1.60, 3.90)], hospital mortality and 90-day mortality were higher (21.1% vs. 4.4%, 52.6% vs. 13.1%), the length of ICU stay was longer [days: 6.0 (4.0, 9.5) vs. 3.0 (1.0, 6.0)], and renal replacement therapy requirement was higher (22.8% vs. 1.9%), with statistically significant differences (all P < 0.05). ROC curve analysis showed that the areas under ROC curve (AUC) of sCys C and uNAG in predicting severe AKI were 0.857 [95% confidence interval (95%CI) was 0.809-0.897)] and 0.735 (95%CI was 0.678-0.788), and the best cut-off values were 1.05 mg/L and 5.39 U/mmol Cr, respectively. The structure of the biomarker decision tree model constructed by biomarkers were intuitive. The overall accuracy in predicting severe AKI was 86.0%, and AUC was 0.905 (95%CI was 0.863-0.937), the sensitivity was 0.912, and the specificity was 0.796. In validation cohort of 130 patients, this decision tree yielded an excellent AUC of 0.909 (95%CI was 0.846-0.952), the sensitivity was 0.906, and the specificity was 0.816, with an overall accuracy of 81.0%. CONCLUSIONS: The decision tree model based on biomarkers for predicting severe AKI in critical patients is highly accurate, intuitive and executable, which is helpful for clinical judgment and decision.

Highly Secure Physically Unclonable Cryptographic Primitives Based on Interfacial Magnetic Anisotropy.

Information security is of great importance for the approaching Internet of things (IoT) era. Physically unclonable functions (PUFs) have been intensively studied for information security. However, silicon PUFs are vulnerable to hazards such as modeling and side-channel attacks. Here we demonstrate a magnetic analogue PUF based on perpendicularly magnetized Ta/CoFeB/MgO heterostructures. The perpendicular magnetic anisotropy originates from the CoFeB/MgO interface, which is sensitive to the subnanometer variation of MgO thickness within a certain range (0.6-1.3 nm). When the MgO layer is thinned, a thickness variation resulting from ion milling nonuniformity induces unclonable random distributions of eas y-axis magnetization orientations in heterostructures. The analogue PUF can provide a much larger key size than a conventional binary-bit counterpart. Moreover, after the thinning process, the unique eas y-axis magnetization orientation in each single device was formed, which can avoid setting random states to realize low power consumption and high-density integration. This magnetic PUF is a promising innovative primitive for secret key generation and storage with high security in the IoT era.