Human skill knowledge guided global trajectory policy reinforcement learning method.

Traditional trajectory learning methods based on Imitation Learning (IL) only learn the existing trajectory knowledge from human demonstration. In this way, it can not adapt the trajectory knowledge to the task environment by interacting with the environment and fine-tuning the policy. To address this problem, a global trajectory learning method which combinines IL with Reinforcement Learning (RL) to adapt the knowledge policy to the environment is proposed. In this paper, IL is proposed to acquire basic trajectory skills, and then learns the agent will explore and exploit more policy which is applicable to the current environment by RL. The basic trajectory skills include the knowledge policy and the time stage information in the whole task space to help learn the time series of the trajectory, and are used to guide the subsequent RL process. Notably, neural networks are not used to model the action policy and the Q value of RL during the RL process. Instead, they are sampled and updated in the whole task space and then transferred to the networks after the RL process through Behavior Cloning (BC) to get continuous and smooth global trajectory policy. The feasibility and the effectiveness of the method was validated in a custom Gym environment of a flower drawing task. And then, we executed the learned policy in the real-world robot drawing experiment.

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation.

Introduction: Behavioral Cloning (BC) is a common imitation learning method which utilizes neural networks to approximate the demonstration action samples for task manipulation skill learning. However, in the real world, the demonstration trajectories from human are often sparse and imperfect, which makes it challenging to comprehensively learn directly from the demonstration action samples. Therefore, in this paper, we proposes a streamlined imitation learning method under the terse geometric representation to take good advantage of the demonstration data, and then realize the manipulation skill learning of assembly tasks. Methods: We map the demonstration trajectories into the geometric feature space. Then we align the demonstration trajectories by Dynamic Time Warping (DTW) method to get the unified data sequence so we can segment them into several time stages. The Probability Movement Primitives (ProMPs) of the demonstration trajectories are then extracted, so we can generate a lot of task trajectories to be the global strategy action samples for training the neural networks. Notalby, we regard the current state of the assembly task as the via point of the ProMPs model to get the generated trajectories, while the time point of the via point is calculated according to the probability model of the different time stages. And we get the action of the current state according to the target position of the next time state. Finally, we train the neural network to obtain the global assembly strategy by Behavioral Cloning. Results: We applied the proposed method to the peg-in-hole assembly task in the simulation environment based on Pybullet + Gym to test its task skill learning performance. And the learned assembly strategy was also executed on a real robotic platform to verify the feasibility of the method further. Discussion: According to the result of the experiment, the proposed method achieves higher success rates compared to traditional imitation learning methods while exhibiting reasonable generalization capabilities. It shows that the ProMPs under geometric representation can help the BC method make better use of the demonstration trajectory and thus better learn the task skills.

Clinical effect of intravascular interventional therapy in the treatment of acute ischemic stroke and its influence on cognitive function, cerebral hemodynamics and inflammatory factors.

Objectives: To evaluate the clinical effect of intravascular interventional therapy in the treatment of acute ischemic stroke and its influence on cognitive function, cerebral hemodynamics and inflammatory factors as well as its clinical significance. Methods: Eighty patients with acute ischemic stroke admitted to Tangxian people's Hospital from January 2018 to January 2020 were randomly divided into two groups. Patients in the control group were treated with conventional thrombolytic therapy on the basis of basic treatment, while patients in the experimental group were given intravascular interventional treatment on the basis of conventional treatment. Clinical effect, recovery of cognitive function and activities of daily living, improvement of cerebral hemodynamics indexes, and changes of inflammatory factors before and after treatment were analyzed by combining NIHSS score and symptom improvement before and two months after treatment, respectively. Results: The effective rate of the experimental group was 82.5% after treatment, and that of the control group was 60%, with a statistically significant difference (P =0.02). Mini-mental State Examination (MMSE) and Barthel scores of the experimental group were significantly improved compared with those of the control group after treatment, with statistically significant differences between the two groups (P=0.00). The mean cerebral vascular blood flow (Qm) of the experimental group increased significantly after treatment compared with that of the control group, while vascular characteristic impedance (ZCV) and peripheral resistance (Rv) decreased significantly in the experimental, with a statistically significant difference (ZCV, P=0.01; Rv, P=0.05); After treatment, TNF- A, CRP, IL-6 and other indicators in the experimental group were significantly lower than those in the control group, with statistically significant differences (P=0.00). Conclusions: Intravascular interventional therapy is an effective treatment regimen for acute ischemic stroke boasting a variety of advantages over conventional thrombolysis, such as significant symptom improvement, high efficacy, favourable recovery of cognitive function and activities of daily living, improvement of cerebral hsemodynamic indicators, and significant reduction of inflammatory cytokine levels.

Creating Better Collision-Free Trajectory for Robot Motion Planning by Linearly Constrained Quadratic Programming.

Many algorithms in probabilistic sampling-based motion planning have been proposed to create a path for a robot in an environment with obstacles. Due to the randomness of sampling, they can efficiently compute the collision-free paths made of segments lying in the configuration space with probabilistic completeness. However, this property also makes the trajectories have some unnecessary redundant or jerky motions, which need to be optimized. For most robotics applications, the trajectories should be short, smooth and keep away from obstacles. This paper proposes a new trajectory optimization technique which transforms a polygon collision-free path into a smooth path, and can deal with trajectories which contain various task constraints. The technique removes redundant motions by quadratic programming in the parameter space of trajectory, and converts collision avoidance conditions to linear constraints to ensure absolute safety of trajectories. Furthermore, the technique uses a projection operator to realize the optimization of trajectories which are subject to some hard kinematic constraints, like keeping a glass of water upright or coordinating operation with dual robots. The experimental results proved the feasibility and effectiveness of the proposed method, when it is compared with other trajectory optimization methods.

Bowel movement frequency and risks of major vascular and non-vascular diseases: a population-based cohort study among Chinese adults.

OBJECTIVE: The application of bowel movement frequency (BMF) in primary care is limited by the lack of solid evidence about the associations of BMF with health outcomes apart from Parkinson's disease and colorectal cancer. We examined the prospective associations of BMF with major vascular and non-vascular diseases outside the digestive system. DESIGN: Population-based prospective cohort study. SETTING: The China Kadoorie Biobank in which participants from 10 geographically diverse areas across China were enrolled between 2004 and 2008. PARTICIPANTS: 487 198 participants aged 30 to 79 years without cancer, heart disease or stroke at baseline were included and followed up for a median of 10 years. The usual BMF was self-reported once at baseline. PRIMARY AND SECONDARY OUTCOME MEASURES: Incident events of predefined major vascular and non-vascular diseases. RESULTS: In multivariable-adjusted analyses, participants having bowel movements 'more than once a day' had higher risks of ischaemic heart disease (IHD), heart failure, chronic obstructive pulmonary disease, type 2 diabetes mellitus and chronic kidney disease (CKD) when compared with the reference group ('once a day'). The respective HRs (95% CIs) were 1.12 (1.09 to 1.16), 1.33 (1.22 to 1.46), 1.28 (1.22 to 1.36), 1.20 (1.15 to 1.26) and 1.15 (1.07 to 1.24). The lowest BMF ('less than three times a week') was also associated with higher risks of IHD, major coronary events, ischaemic stroke and CKD. The respective HRs were 1.07 (1.02 to 1.12), 1.22 (1.10 to 1.36), 1.11 (1.05 to 1.16) and 1.20 (1.07 to 1.35). CONCLUSION: BMF was associated with future risks of multiple vascular and non-vascular diseases. The integration of BMF assessment and health counselling into primary care should be considered.

Diabetes, Plasma Glucose, and Incidence of Fatty Liver, Cirrhosis, and Liver Cancer: A Prospective Study of 0.5 Million People.

The prevalence of diabetes is increasing rapidly in China. However, evidence is limited about its effects on chronic liver diseases and liver cancer. We examined the associations of diabetes with chronic liver diseases and liver cancer and of random plasma glucose (RPG) with these liver diseases among participants without diabetes in Chinese adults and the possible interaction by hepatitis B virus (HBV) infection. The prospective China Kadoorie Biobank recruited 512,891 adults. During 10 years of follow-up, 2,568 liver cancer, 2,082 cirrhosis, 1,298 hospitalized nonalcoholic fatty liver disease (NAFLD), and 244 hospitalized alcoholic liver disease (ALD) cases were recorded among 503,993 participants without prior history of cancer or chronic liver diseases at baseline. Cox regression was used to estimate hazard ratios (HRs) for each disease by diabetes status (previously diagnosed or screen-detected) and, among those without previously diagnosed diabetes, by levels of RPG. Overall 5.8% of participants had diabetes at baseline. Compared to those without diabetes, individuals with diabetes had adjusted HRs of 1.49 (95% confidence interval 1.30-1.70) for liver cancer, 1.81 (1.57-2.09) for cirrhosis, 1.76 (1.47-2.16) for NAFLD, and 2.24 (1.42-3.54) for ALD. The excess risks decreased but remained elevated in those with longer duration. Among those without previously diagnosed diabetes, RPG was positively associated with liver diseases, with adjusted HRs per 1 mmol/L higher RPG of 1.04 (1.03-1.06) for liver cancer, 1.07 (1.05-1.09) for cirrhosis, 1.07 (1.05-1.10) for NAFLD, and 1.10 (1.05-1.15) for ALD. These associations did not differ by HBV infection. CONCLUSION: In Chinese adults, diabetes and higher blood glucose levels among those without known diabetes are associated with higher risks of liver cancer and major chronic liver diseases.