Language inference-based learning for Low-Resource Chinese clinical named entity recognition using language model.

Electronic health records (EHRs) have been widely used and are gradually replacing paper records. Therefore, extracting valuable information from EHRs has become the focus and hotspot of current research. Clinical named entity recognition (CNER) is an important task in information extraction. Most current research methods used standard supervised learning approaches to fine-tune pre-trained language models (PLMs), which require a large amount of annotated data for model training. However, in realistic medical scenarios, annotated data are scarce, especially in the healthcare field. The process of annotating data in real clinical settings is time-consuming and labour-intensive. In this paper, a language inference-based learning method (LANGIL) is proposed to study clinical NER tasks with limited annotated samples, i.e., in low-resource clinical scenarios. A method based on prompt learning is designed to reformulate the entity recognition task into a language inference-based task. Differing from the standard fine-tuning method, the approach introduced in this paper does not design the additional network layers that train from scratch. This alleviates the gap between pre-training tasks and downstream tasks, allowing the comprehension capabilities of PLMs to be leveraged under the condition of limited training samples. The experiments on four Chinese clinical named entity recognition datasets showed that LANGIL achieves significant improvements in F1-score compared to the former method.

An intelligent adverse delivery outcomes prediction model based on the fusion of multiple obstetric clinical data.

Adverse delivery outcomes is a major re-productive health problem that affects the physical and mental health of pregnant women. Obviously, obstetric clinical data has periodically time series characteristics. This paper proposed a three stage adverse delivery outcomes prediction model via the fusion of multiple time series clinical data. The first stage is data aggregation, in which the data set is collected from the obstetric clinical data and divided based on time series features. In the second stage, a multi-channel gated cycle unit is used to solve the calculation error caused by irregular sampling of time series data. The hidden layer feature vector is connected with the fully connected layer, reshaped into a new one-dimensional feature, and fused with the non-time series data into a new data set. The third stage is the prediction stage of adverse delivery outcomes. By connecting the multi-channel gated cycle unit with the extreme gradient lift, the data transmitted in the corresponding channel is used in the feature extraction stage, in which the weighted entropy-based feature extraction is adopted. With the help of the extracted features, a hybrid artificial neural network architecture (MGRU-XGB) was developed to predict adverse delivery outcomes. The experimental results showed that the hybrid model had the best prediction performance for adverse delivery outcomes compared with other single models in terms of sensitivity, specificity, AUC and other evaluation indexes.

Learning implicit sentiments in Alzheimer's disease recognition with contextual attention features.

Background: Alzheimer's disease (AD) is difficult to diagnose on the basis of language because of the implicit emotion of transcripts, which is defined as a supervised fuzzy implicit emotion classification at the document level. Recent neural network-based approaches have not paid attention to the implicit sentiments entailed in AD transcripts. Method: A two-level attention mechanism is proposed to detect deep semantic information toward words and sentences, which enables it to attend to more words and fewer sentences differentially when constructing document representation. Specifically, a document vector was built by progressively aggregating important words into sentence vectors and important sentences into document vectors. Results: Experimental results showed that our method achieved the best accuracy of 91.6% on annotated public Pitt corpora, which validates its effectiveness in learning implicit sentiment representation for our model. Conclusion: The proposed model can qualitatively select informative words and sentences using attention layers, and this method also provides good inspiration for AD diagnosis based on implicit sentiment transcripts.

An LBS and agent-based simulator for Covid-19 research.

The mobility data of citizens provide important information on the epidemic spread including Covid-19. However, the privacy versus security dilemma hinders the utilization of such data. This paper proposed a method to generate pseudo mobility data on a per-agent basis, utilizing the actual geographical environment data provided by LBS to generate the agent-specific mobility trajectories and export them as GPS-like data. Demographic characteristics such as behavior patterns, gender, age, vaccination, and mask-wearing status are also assigned to the agents. A web-based data generator was implemented, enabling users to make detailed settings to meet different research needs. The simulated data indicated the usability of the proposed methods.

The Prediction of Preterm Birth Using Time-Series Technology-Based Machine Learning: Retrospective Cohort Study.

BACKGROUND: Globally, the preterm birth rate has tended to increase over time. Ultrasonography cervical-length assessment is considered to be the most effective screening method for preterm birth, but routine, universal cervical-length screening remains controversial because of its cost. OBJECTIVE: We used obstetric data to analyze and assess the risk of preterm birth. A machine learning model based on time-series technology was used to analyze regular, repeated obstetric examination records during pregnancy to improve the performance of the preterm birth screening model. METHODS: This study attempts to use continuous electronic medical record (EMR) data from pregnant women to construct a preterm birth prediction classifier based on long short-term memory (LSTM) networks. Clinical data were collected from 5187 pregnant Chinese women who gave birth with natural vaginal delivery. The data included more than 25,000 obstetric EMRs from the early trimester to 28 weeks of gestation. The area under the curve (AUC), accuracy, sensitivity, and specificity were used to assess the performance of the prediction model. RESULTS: Compared with a traditional cross-sectional study, the LSTM model in this time-series study had better overall prediction ability and a lower misdiagnosis rate at the same detection rate. Accuracy was 0.739, sensitivity was 0.407, specificity was 0.982, and the AUC was 0.651. Important-feature identification indicated that blood pressure, blood glucose, lipids, uric acid, and other metabolic factors were important factors related to preterm birth. CONCLUSIONS: The results of this study will be helpful to the formulation of guidelines for the prevention and treatment of preterm birth, and will help clinicians make correct decisions during obstetric examinations. The time-series model has advantages for preterm birth prediction.

A Transfer Learning Method for Detecting Alzheimer's Disease Based on Speech and Natural Language Processing.

Alzheimer's disease (AD) is a neurodegenerative disease that is difficult to be detected using convenient and reliable methods. The language change in patients with AD is an important signal of their cognitive status, which potentially helps in early diagnosis. In this study, we developed a transfer learning model based on speech and natural language processing (NLP) technology for the early diagnosis of AD. The lack of large datasets limits the use of complex neural network models without feature engineering, while transfer learning can effectively solve this problem. The transfer learning model is firstly pre-trained on large text datasets to get the pre-trained language model, and then, based on such a model, an AD classification model is performed on small training sets. Concretely, a distilled bidirectional encoder representation (distilBert) embedding, combined with a logistic regression classifier, is used to distinguish AD from normal controls. The model experiment was evaluated on Alzheimer's dementia recognition through spontaneous speech datasets in 2020, including the balanced 78 healthy controls (HC) and 78 patients with AD. The accuracy of the proposed model is 0.88, which is almost equivalent to the champion score in the challenge and a considerable improvement over the baseline of 75% established by organizers of the challenge. As a result, the transfer learning method in this study improves AD prediction, which does not only reduces the need for feature engineering but also addresses the lack of sufficiently large datasets.

Estimated date of delivery with electronic medical records by a hybrid GBDT-GRU model.

An accurate estimated date of delivery (EDD) helps pregnant women make adequate preparations before delivery and avoid the panic of parturition. EDD is normally derived from some formulates or estimated by doctors based on last menstruation period and ultrasound examinations. This study attempted to combine antenatal examinations and electronic medical records to develop a hybrid model based on Gradient Boosting Decision Tree and Gated Recurrent Unit (GBDT-GRU). Besides exploring the features that affect the EDD, GBDT-GRU model obtained the results by dynamic prediction of different stages. The mean square error (MSE) and coefficient of determination (R2) were used to compare the performance among the different prediction methods. In addition, we evaluated predictive performances of different prediction models by comparing the proportion of pregnant women under the error of different days. Experimental results showed that the performance indexes of hybrid GBDT-GRU model outperformed other prediction methods because it focuses on analyzing the time-series predictors of pregnancy. The results of this study are helpful for the development of guidelines for clinical delivery treatments, as it can assist clinicians in making correct decisions during obstetric examinations.

Behavioral Responses of Pregnant Women to the Early Stage of COVID-19 Pandemic in the Network Era in China: Online Questionnaire Study.

PURPOSE: The aim of this study was to examine the behavioral responses of pregnant women during the early stage of Coronavirus Disease 2019 (COVID-19) outbreak. METHODS: We recruited 1,099 women to complete an online questionnaire survey from February 10 to February 25, 2020. The subjects were divided into two groups (the pregnant women group and the control group). RESULTS: Concerns about infection: most of the participants watched the COVID-19 news at least once a day. Protective behaviors: the utilization rate of pregnant women (often using various measures) was higher than that of nonpregnant women. Exercise: 30.6% of the pregnant women continued to exercise at home, whereas in the control group, this percentage was 8.4%. Spouse relationship: 38.8% of the subjects' relationship improved, whereas only 2.3% thought the relationship was getting worse. CONCLUSION: Pregnant women had some unique behavioral responses different from that of nonpregnant women. It is important to understand the behavioral responses of pregnant women in this network era.

Fetal birthweight prediction with measured data by a temporal machine learning method.

BACKGROUND: Birthweight is an important indicator during the fetal development process to protect the maternal and infant safety. However, birthweight is difficult to be directly measured, and is usually roughly estimated by the empirical formulas according to the experience of the doctors in clinical practice. METHODS: This study attempts to combine multiple electronic medical records with the B-ultrasonic examination of pregnant women to construct a hybrid birth weight predicting classifier based on long short-term memory (LSTM) networks. The clinical data were collected from 5,759 Chinese pregnant women who have given birth, with more than 57,000 obstetric electronic medical records. We evaluated the prediction by the mean relative error (MRE) and the accuracy rate of different machine learning classifiers at different predicting periods for first delivery and multiple deliveries. Additionally, we evaluated the classification accuracies of different classifiers respectively for the Small-for-Gestational-age (SGA), Large-for-Gestational-Age (LGA) and Appropriate-for-Gestational-Age (AGA) groups. RESULTS: The results show that the accuracy rate of the prediction model using Convolutional Neuron Networks (CNN), Random Forest (RF), Linear-Regression, Support Vector Regression (SVR), Back Propagation Neural Network(BPNN), and the proposed hybrid-LSTM at the 40th pregnancy week for first delivery were 0.498, 0.662, 0.670, 0.680, 0.705 and 0.793, respectively. Among the groups of less than 39th pregnancy week, the 39th pregnancy week and more than 40th week, the hybrid-LSTM model obtained the best accuracy and almost the least MRE compared with those of machine learning models. Not surprisingly, all the machine learning models performed better than the empirical formula. In the SGA, LGA and AGA group experiments, the average accuracy by the empirical formula, logistic regression (LR), BPNN, CNN, RF and Hybrid-LSTM were 0.780, 0.855, 0.890, 0.906, 0.916 and 0.933, respectively. CONCLUSIONS: The results of this study are helpful for the birthweight prediction and development of guidelines for clinical delivery treatments. It is also useful for the implementation of a decision support system using the temporal machine learning prediction model, as it can assist the clinicians to make correct decisions during the obstetric examinations and remind pregnant women to manage their weight.

Improving Alzheimer's Disease Detection for Speech Based on Feature Purification Network.

Alzheimer's disease (AD) is a neurodegenerative disease involving the decline of cognitive ability with illness progresses. At present, the diagnosis of AD mainly depends on the interviews between patients and doctors, which is slow, expensive, and subjective, so it is not a better solution to recognize AD using the currently available neuropsychological examinations and clinical diagnostic criteria. A recent study has indicated the potential of language analysis for AD diagnosis. In this study, we proposed a novel feature purification network that can improve the representation learning of transformer model further. Though transformer has made great progress in generating discriminative features because of its long-distance reasoning ability, there is still room for improvement. There exist many common features that are not indicative of any specific class, and we rule out the influence of common features from traditional features extracted by transformer encoder and can get more discriminative features for classification. We apply this method to improve transformer's performance on three public dementia datasets and get improved classification results markedly. Specifically, the method on Pitt datasets gets state-of-the-art (SOTA) result.

Medical Knowledge Extraction and Analysis from Electronic Medical Records Using Deep Learning.

Objectives Medical knowledge extraction (MKE) plays a key role in natural language processing (NLP) research in electronic medical records (EMR), which are the important digital carriers for recording medical activities of patients. Named entity recognition (NER) and medical relation extraction (MRE) are two basic tasks of MKE. This study aims to improve the recognition accuracy of these two tasks by exploring deep learning methods. Methods This study discussed and built two application scenes of bidirectional long short-term memory combined conditional random field (BiLSTM-CRF) model for NER and MRE tasks. In the data preprocessing of both tasks, a GloVe word embedding model was used to vectorize words. In the NER task, a sequence labeling strategy was used to classify each word tag by the joint probability distribution through the CRF layer. In the MRE task, the medical entity relation category was predicted by transforming the classification problem of a single entity into a sequence classification problem and linking the feature combinations between entities also through the CRF layer. Results Through the validation on the I2B2 2010 public dataset, the BiLSTM-CRF models built in this study got much better results than the baseline methods in the two tasks, where the F1-measure was up to 0.88 in NER task and 0.78 in MRE task. Moreover, the model converged faster and avoided problems such as overfitting. Conclusion This study proved the good performance of deep learning on medical knowledge extraction. It also verified the feasibility of the BiLSTM-CRF model in different application scenarios, laying the foundation for the subsequent work in the EMR field.

A localized Richardson-Lucy algorithm for fiber orientation estimation in high angular resolution diffusion imaging.

PURPOSE: Diffusion tensor imaging is widely used for studying neural fiber trajectories in white matter and for quantifying changes in tissue using diffusion properties at each voxel in the brain. To better model the nature of crossing fibers within complex architectures, rather than using a simplified tensor model that assumes only a single fiber direction at each image voxel, a model mixing multiple diffusion tensors is used to profile diffusion signals from high angular resolution diffusion imaging (HARDI) data. Based on the HARDI signal and a multiple tensors model, spherical deconvolution methods have been developed to overcome the limitations of the diffusion tensor model when resolving crossing fibers. The Richardson-Lucy algorithm is a popular spherical deconvolution method used in previous work. However, it is based on a Gaussian distribution, while HARDI data are always very noisy, and the distribution of HARDI data follows a Rician distribution. This current work aims to present a novel solution to address these issues. METHODS: By simultaneously considering both the Rician bias and neighbor correlation in HARDI data, the authors propose a localized Richardson-Lucy (LRL) algorithm to estimate fiber orientations for HARDI data. The proposed method can simultaneously reduce noise and correct the Rician bias. RESULTS: Mean angular error (MAE) between the estimated Fiber orientation distribution (FOD) field and the reference FOD field was computed to examine whether the proposed LRL algorithm offered any advantage over the conventional RL algorithm at various levels of noise. Normalized mean squared error (NMSE) was also computed to measure the similarity between the true FOD field and the estimated FOD filed. For MAE comparisons, the proposed LRL approach obtained the best results in most of the cases at different levels of SNR and b-values. For NMSE comparisons, the proposed LRL approach obtained the best results in most of the cases at b-value = 3000 s/mm(2), which is the recommended schema for HARDI data acquisition. In addition, the FOD fields estimated by the proposed LRL approach in regions of fiber crossing regions using real data sets also showed similar fiber structures which agreed with common acknowledge in these regions. CONCLUSIONS: The novel spherical deconvolution method for improved accuracy in investigating crossing fibers can simultaneously reduce noise and correct Rician bias. With the noise smoothed and bias corrected, this algorithm is especially suitable for estimation of fiber orientations in HARDI data. Experimental results using both synthetic and real imaging data demonstrated the success and effectiveness of the proposed LRL algorithm.

Medical image registration by combining global and local information: a chain-type diffeomorphic demons algorithm.

The demons algorithm is a popular algorithm for non-rigid image registration because of its computational efficiency and simple implementation. The deformation forces of the classic demons algorithm were derived from image gradients by considering the deformation to decrease the intensity dissimilarity between images. However, the methods using the difference of image intensity for medical image registration are easily affected by image artifacts, such as image noise, non-uniform imaging and partial volume effects. The gradient magnitude image is constructed from the local information of an image, so the difference in a gradient magnitude image can be regarded as more reliable and robust for these artifacts. Then, registering medical images by considering the differences in both image intensity and gradient magnitude is a straightforward selection. In this paper, based on a diffeomorphic demons algorithm, we propose a chain-type diffeomorphic demons algorithm by combining the differences in both image intensity and gradient magnitude for medical image registration. Previous work had shown that the classic demons algorithm can be considered as an approximation of a second order gradient descent on the sum of the squared intensity differences. By optimizing the new dissimilarity criteria, we also present a set of new demons forces which were derived from the gradients of the image and gradient magnitude image. We show that, in controlled experiments, this advantage is confirmed, and yields a fast convergence.