Automatic knowledge extraction from Chinese electronic medical records and rheumatoid arthritis knowledge graph construction.

Background: Knowledge graphs are a powerful tool for organizing knowledge, processing information and integrating scattered information, effectively visualizing the relationships among entities and supporting further intelligent applications. One of the critical tasks in building knowledge graphs is knowledge extraction. The existing knowledge extraction models in the Chinese medical domain usually require high-quality and large-scale manually labeled corpora for model training. In this study, we investigate rheumatoid arthritis (RA)-related Chinese electronic medical records (CEMRs) and address the automatic knowledge extraction task with a small number of annotated samples from CEMRs, from which an authoritative RA knowledge graph is constructed. Methods: After constructing the domain ontology of RA and completing manual labeling, we propose the MC-bidirectional encoder representation from transformers-bidirectional long short-term memory-conditional random field (BERT-BiLSTM-CRF) model for the named entity recognition (NER) task and the MC-BERT + feedforward neural network (FFNN) model for the entity extraction task. The pretrained language model (MC-BERT) is trained with many unlabeled medical data and fine-tuned using other medical domain datasets. We apply the established model to automatically label the remaining CEMRs, and then an RA knowledge graph is constructed based on the entities and entity relations, a preliminary assessment is conducted, and an intelligent application is presented. Results: The proposed model achieved better performance than that of other widely used models in knowledge extraction tasks, with mean F1 scores of 92.96% in entity recognition and 95.29% in relation extraction. This study preliminarily confirmed that using a pretrained medical language model could solve the problem that knowledge extraction from CEMRs requires a large number of manual annotations. An RA knowledge graph based on the above identified entities and extracted relations from 1,986 CEMRs was constructed. Experts verified the effectiveness of the constructed RA knowledge graph. Conclusions: In this paper, an RA knowledge graph based on CEMRs was established, the processes of data annotation, automatic knowledge extraction, and knowledge graph construction were described, and a preliminary assessment and an application were presented. The study demonstrated the viability of a pretrained language model combined with a deep neural network for knowledge extraction tasks from CEMRs based on a small number of manually annotated samples.

Chinese Clinical Named Entity Recognition From Electronic Medical Records Based on Multisemantic Features by Using Robustly Optimized Bidirectional Encoder Representation From Transformers Pretraining Approach Whole Word Masking and Convolutional Neural Networks: Model Development and Validation.

BACKGROUND: Clinical electronic medical records (EMRs) contain important information on patients' anatomy, symptoms, examinations, diagnoses, and medications. Large-scale mining of rich medical information from EMRs will provide notable reference value for medical research. With the complexity of Chinese grammar and blurred boundaries of Chinese words, Chinese clinical named entity recognition (CNER) remains a notable challenge. Follow-up tasks such as medical entity structuring, medical entity standardization, medical entity relationship extraction, and medical knowledge graph construction largely depend on medical named entity recognition effects. A promising CNER result would provide reliable support for building domain knowledge graphs, knowledge bases, and knowledge retrieval systems. Furthermore, it would provide research ideas for scientists and medical decision-making references for doctors and even guide patients on disease and health management. Therefore, obtaining excellent CNER results is essential. OBJECTIVE: We aimed to propose a Chinese CNER method to learn semantics-enriched representations for comprehensively enhancing machines to understand deep semantic information of EMRs by using multisemantic features, which makes medical information more readable and understandable. METHODS: First, we used Robustly Optimized Bidirectional Encoder Representation from Transformers Pretraining Approach Whole Word Masking (RoBERTa-wwm) with dynamic fusion and Chinese character features, including 5-stroke code, Zheng code, phonological code, and stroke code, extracted by 1-dimensional convolutional neural networks (CNNs) to obtain fine-grained semantic features of Chinese characters. Subsequently, we converted Chinese characters into square images to obtain Chinese character image features from another modality by using a 2-dimensional CNN. Finally, we input multisemantic features into Bidirectional Long Short-Term Memory with Conditional Random Fields to achieve Chinese CNER. The effectiveness of our model was compared with that of the baseline and existing research models, and the features involved in the model were ablated and analyzed to verify the model's effectiveness. RESULTS: We collected 1379 Yidu-S4K EMRs containing 23,655 entities in 6 categories and 2007 self-annotated EMRs containing 118,643 entities in 7 categories. The experiments showed that our model outperformed the comparison experiments, with F1-scores of 89.28% and 84.61% on the Yidu-S4K and self-annotated data sets, respectively. The results of the ablation analysis demonstrated that each feature and method we used could improve the entity recognition ability. CONCLUSIONS: Our proposed CNER method would mine the richer deep semantic information in EMRs by multisemantic embedding using RoBERTa-wwm and CNNs, enhancing the semantic recognition of characters at different granularity levels and improving the generalization capability of the method by achieving information complementarity among different semantic features, thus making the machine semantically understand EMRs and improving the CNER task accuracy.

Using Semantic Text Similarity calculation for question matching in a rheumatoid arthritis question-answering system.

Background: When users inquire about knowledge in a certain field using the internet, the intelligent question-answering system based on frequently asked questions (FAQs) provides numerous concise and accurate answers that have been manually verified. However, there are few specific question-answering systems for chronic diseases, such as rheumatoid arthritis, and the related technology to construct a question-answering system about chronic diseases is not sufficiently mature. Methods: Our research embedded the classification information of the question into the sentence vector based on the bidirectional encoder representations from transformers (BERT) language model. First of all, we calculated the similarity using edit distance to recall the candidate set of similar questions. Then, we took advantage of the BERT pretraining model to map the sentence information to the corresponding embedding representation. Finally, each dimensional feature of the sentence was obtained by passing a sentence vector through the multihead attention layer and the fully connected feedforward layer. The features that were stitched and fused were used for the semantic similarity calculation. Results: Our improved model achieved a Top-1 precision of 0.551, Top-3 precision of 0.767, and Top-5 precision of 0.813 on 176 testing question sentences. In the analysis of the actual application effect of the model, we found that our model performed well in understanding the actual intention of users. Conclusions: Our deep learning model takes into account the background and classifications of questions and combines the efficiency of deep learning technology and the comprehensibility of semantics. The model enables the deep meaning of the user's question to be better understood by the intelligent question answering system, and answers that are more relevant to the original query are provided.

Surgical procedure long terms recognition from Chinese literature incorporating structural feature.

With rapid development of technologies in medical diagnosis and treatment, the novel and complicated concepts and usages of clinical terms especially of surgical procedures have become common in daily routine. Expected to be performed in an operating room and accompanied by an incision based on expert discretion, surgical procedures imply clinical understanding of diagnosis, examination, testing, equipment, drugs and symptoms, etc., but terms expressing surgical procedures are difficult to recognize since the terms are highly distinctive due to long morphological length and complex linguistics phenomena. To achieve higher recognition performance and overcome the challenge of the absence of natural delimiters in Chinese sentences, we propose a Named Entity Recognition (NER) model named Structural-SoftLexicon-Bi-LSTM-CRF (SSBC) empowered by pre-trained model BERT. In particular, we pre-trained a lexicon embedding over large-scale medical corpus to better leverage domain-specific structural knowledge. With input additionally augmented by BERT, rich multigranular information and structural term information is transferred from Structural-SoftLexicon to downstream model Bi-LSTM-CRF. Therefore, we could get a global optimal prediction of input sequence. We evaluate our model on a self-built corpus and results show that SSBC with pre-trained model outperforms other state-of-the-art benchmarks, surpassing at most 3.77% in F1 score. This study hopefully would benefit Diagnostic Related Groups (DRGs) and Diagnosis Intervention Package (DIP) grouping system, medical records statistics and analysis, Medicare payment system, etc.

Multi-Label Classification in Patient-Doctor Dialogues With the RoBERTa-WWM-ext + CNN (Robustly Optimized Bidirectional Encoder Representations From Transformers Pretraining Approach With Whole Word Masking Extended Combining a Convolutional Neural Network) Model: Named Entity Study.

BACKGROUND: With the prevalence of online consultation, many patient-doctor dialogues have accumulated, which, in an authentic language environment, are of significant value to the research and development of intelligent question answering and automated triage in recent natural language processing studies. OBJECTIVE: The purpose of this study was to design a front-end task module for the network inquiry of intelligent medical services. Through the study of automatic labeling of real doctor-patient dialogue text on the internet, a method of identifying the negative and positive entities of dialogues with higher accuracy has been explored. METHODS: The data set used for this study was from the Spring Rain Doctor internet online consultation, which was downloaded from the official data set of Alibaba Tianchi Lab. We proposed a composite abutting joint model, which was able to automatically classify the types of clinical finding entities into the following 4 attributes: positive, negative, other, and empty. We adapted a downstream architecture in Chinese Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach (RoBERTa) with whole word masking (WWM) extended (RoBERTa-WWM-ext) combining a text convolutional neural network (CNN). We used RoBERTa-WWM-ext to express sentence semantics as a text vector and then extracted the local features of the sentence through the CNN, which was our new fusion model. To verify its knowledge learning ability, we chose Enhanced Representation through Knowledge Integration (ERNIE), original Bidirectional Encoder Representations from Transformers (BERT), and Chinese BERT with WWM to perform the same task, and then compared the results. Precision, recall, and macro-F1 were used to evaluate the performance of the methods. RESULTS: We found that the ERNIE model, which was trained with a large Chinese corpus, had a total score (macro-F1) of 65.78290014, while BERT and BERT-WWM had scores of 53.18247117 and 69.2795315, respectively. Our composite abutting joint model (RoBERTa-WWM-ext + CNN) had a macro-F1 value of 70.55936311, showing that our model outperformed the other models in the task. CONCLUSIONS: The accuracy of the original model can be greatly improved by giving priority to WWM and replacing the word-based mask with unit to classify and label medical entities. Better results can be obtained by effectively optimizing the downstream tasks of the model and the integration of multiple models later on. The study findings contribute to the translation of online consultation information into machine-readable information.

Model-based clinical note entity recognition for rheumatoid arthritis using bidirectional encoder representation from transformers.

BACKGROUND: Rheumatoid arthritis (RA) is a disease of the immune system with a high rate of disability and there are a large amount of valuable disease diagnosis and treatment information in the clinical note of the electronic medical record. Artificial intelligence methods can be used to mine useful information in clinical notes effectively. This study aimed to develop an effective method to identify and classify medical entities in the clinical notes relating to RA and use the entity identification results in subsequent studies. METHODS: In this paper, we introduced the bidirectional encoder representation from transformers (BERT) pre-training model to enhance the semantic representation of word vectors. The generated word vectors were then inputted into the model, which is composed of traditional bidirectional long short-term memory neural networks and conditional random field machine learning algorithms for the named entity recognition of clinical notes to improve the model's effectiveness. The BERT method takes the combination of token embeddings, segment embeddings, and position embeddings as the model input and fine-tunes the model during training. RESULTS: Compared with the traditional Word2vec word vector model, the performance of the BERT pre-training model to obtain a word vector as model input was significantly improved. The best F1-score of the named entity recognition task after training using many rheumatoid arthritis clinical notes was 0.936. CONCLUSIONS: This paper confirms the effectiveness of using an advanced artificial intelligence method to carry out named entity recognition tasks on a corpus of a large number of clinical notes; this application is promising in the medical setting. Moreover, the extraction of results in this study provides a lot of basic data for subsequent tasks, including relation extraction, medical knowledge graph construction, and disease reasoning.

Analysis of diverse factors influencing the health status as well as medical and health service utilization in the floating elderly of China.

BACKGROUND: Based on the "China Migrants Dynamic Survey-Special investigation on Floating Elderly in 8 megacities in 2015", the health status and the utilization of medical and health services in floating elderly were described and analyzed. OBJECTIVE: Scientific basis and critical suggestions are provided for improving the utilization level of medical and health services in the floating elderly and designing targeted health policies to improve their well-being. METHODS: The rank-sum test and Pearson χ2 test were used to compare the health status of floating elderly with different characteristics. Thereafter based on Andersen model, floating characteristics were added and binary logistic regression was used to explore the influencing factors of medical and health service utilization in the floating elderly. RESULTS: About 94.7% of the floating elderly were self-assessed as healthy/basically healthy. About 24.2% had hypertension or diabetes as diagnosed by the qualified doctors. About 7% suffered from diseases that required hospitalization. Only 28.6% of the floating elderly with hypertension or diabetes had visited a doctor for follow-up. In the case of minor ailments, only 48.7% decided to visit the clinics. Approximately 70.7% of the floating elderly had used in-patient services when they suffered from diseases requiring hospitalization. CONCLUSION: The floating elderly were observed to be generally in good health but a high prevalence of hypertension or diabetes was observed among them. The cultivation of health awareness was found to be of great significance contributing to the improvement of the overall health level among the floating elderly. The basic medical insurance coverage was low, and the medical and health services were found to be severely underutilized. Adequate social support can promote the health of the floating elderly and improve their utilization of medical and health services. The floating reasons, scope and years of the elderly significantly affected their health status and the utilization rate of the basic public health services.

Comparison of serum PCT and CRP levels in patients infected by different pathogenic microorganisms: a systematic review and meta-analysis.

To avoid the abuse and misuse of antibiotics, procalcitonin (PCT) and C-reactive protein (CRP) have been used as new approaches to identify different types of infection. Multiple databases were adopted to search relevant studies, and the articles that satisfied the inclusion criteria were included. Meta-analyses were conducted with Review Manager 5.0, and to estimate the quality of each article, risk of bias was assessed. Eight articles satisfied the inclusion criteria. The concentrations of both PCT and CRP in patients with bacterial infection were higher than those with non-bacterial infection. Both PCT and CRP levels in patients with G- bacterial infection were higher than in those with G+ bacterial infection and fungus infection. In the G+ bacterial infection group, a higher concentration of CRP was observed compared with fungus infection group, while the difference of PCT between G+ bacterial infection and fungus infection was not significant. Our study suggested that both PCT and CRP are helpful to a certain extent in detecting pneumonia caused by different types of infection.

Comparison of serum PCT and CRP levels in patients infected by different pathogenic microorganisms: a systematic review and meta-analysis

To avoid the abuse and misuse of antibiotics, procalcitonin (PCT) and C-reactive protein (CRP) have been used as new approaches to identify different types of infection. Multiple databases were adopted to search relevant studies, and the articles that satisfied the inclusion criteria were included. Meta-analyses were conducted with Review Manager 5.0, and to estimate the quality of each article, risk of bias was assessed. Eight articles satisfied the inclusion criteria. The concentrations of both PCT and CRP in patients with bacterial infection were higher than those with non-bacterial infection. Both PCT and CRP levels in patients with G− bacterial infection were higher than in those with G+ bacterial infection and fungus infection. In the G+ bacterial infection group, a higher concentration of CRP was observed compared with fungus infection group, while the difference of PCT between G+ bacterial infection and fungus infection was not significant. Our study suggested that both PCT and CRP are helpful to a certain extent in detecting pneumonia caused by different types of infection.

A Robust and Accurate Non-rigid Medical Image Registration Algorithm Based on Multi-level Deformable Model.

BACKGROUND: Compared to the rigid image registration task, the non-rigid image registration task faces much more challenges due to its high degree of freedom and inherent requirement of smoothness in the deformation field. The purpose was to propose an efficient coarse-to-fine non-rigid medical image registration algorithm based on a multilevel deformable model. METHODS: In this paper, a robust and efficient coarse-to-fine non-rigid medical image registration algorithm is proposed. It contains three level deformation models, i.e., the global homography model, the local mesh-level homography model, and the local B-spline FFD (Free-Form Deformation) model. The coarse registration is achieved by the first two level models. In the global homography model, a robust algorithm for simultaneous outliers (error matched feature points) removal and model estimation is applied. In the local mesh-level homography model, a new similarity measure is proposed to improve the robustness and accuracy of local mesh based registration. In the fine registration, a local B-spline FFD model with normalized mutual information gradient is employed. RESULTS: We verified the effectiveness of each stage of the proposed registration algorithm with many non-rigid transformation image pairs, and quantitatively compared our proposed registration algorithm with the HBFFD method which is based on the control points of multi-resolution. The experimental results show that our algorithm is more accurate than the hierarchical local B-spline FFD method. CONCLUSION: Our algorithm can achieve high precision registration by coarse-to-fine process based on multi-level deformable model, which ourperforms the state-of-the-art methods.