Applying generative AI with retrieval augmented generation to summarize and extract key clinical information from electronic health records.

BACKGROUND: Malnutrition is a prevalent issue in aged care facilities (RACFs), leading to adverse health outcomes. The ability to efficiently extract key clinical information from a large volume of data in electronic health records (EHR) can improve understanding about the extent of the problem and developing effective interventions. This research aimed to test the efficacy of zero-shot prompt engineering applied to generative artificial intelligence (AI) models on their own and in combination with retrieval augmented generation (RAG), for the automating tasks of summarizing both structured and unstructured data in EHR and extracting important malnutrition information. METHODOLOGY: We utilized Llama 2 13B model with zero-shot prompting. The dataset comprises unstructured and structured EHRs related to malnutrition management in 40 Australian RACFs. We employed zero-shot learning to the model alone first, then combined it with RAG to accomplish two tasks: generate structured summaries about the nutritional status of a client and extract key information about malnutrition risk factors. We utilized 25 notes in the first task and 1,399 in the second task. We evaluated the model's output of each task manually against a gold standard dataset. RESULT: The evaluation outcomes indicated that zero-shot learning applied to generative AI model is highly effective in summarizing and extracting information about nutritional status of RACFs' clients. The generated summaries provided concise and accurate representation of the original data with an overall accuracy of 93.25%. The addition of RAG improved the summarization process, leading to a 6% increase and achieving an accuracy of 99.25%. The model also proved its capability in extracting risk factors with an accuracy of 90%. However, adding RAG did not further improve accuracy in this task. Overall, the model has shown a robust performance when information was explicitly stated in the notes; however, it could encounter hallucination limitations, particularly when details were not explicitly provided. CONCLUSION: This study demonstrates the high performance and limitations of applying zero-shot learning to generative AI models to automatic generation of structured summarization of EHRs data and extracting key clinical information. The inclusion of the RAG approach improved the model performance and mitigated the hallucination problem.

Machine Learning Model to Extract Malnutrition Data from Nursing Notes.

Malnutrition is a severe health problem that is prevalent in older people residing in residential aged care facilities. Recent advancements in machine learning have made it possible to extract key insight from electronic health records. To date, few researchers applied these techniques to classify nursing notes automatically. Therefore, we propose a model based on ClinicalBioBert to identify malnutrition notes. We evaluated our approach with two mainstream approaches. Our approach had the highest F1-score of 0.90.

Malnutrition and its contributing factors for older people living in residential aged care facilities: Insights from natural language processing of aged care records.

BACKGROUND: Malnutrition is a serious health risk facing older people living in residential aged care facilities. Aged care staff record observations and concerns about older people in electronic health records (EHR), including free-text progress notes. These insights are yet to be unleashed. OBJECTIVE: This study explored the risk factors for malnutrition in structured and unstructured electronic health data. METHODS: Data of weight loss and malnutrition were extracted from the de-identified EHR records of a large aged care organization in Australia. A literature review was conducted to identify causative factors for malnutrition. Natural language processing (NLP) techniques were applied to progress notes to extract these causative factors. The NLP performance was evaluated by the parameters of sensitivity, specificity and F1-Score. RESULTS: The NLP methods were highly accurate in extracting the key data, values for 46 causative variables, from the free-text client progress notes. Thirty three percent (1,469 out of 4,405) of the clients were malnourished. The structured, tabulated data only recorded 48% of these malnourished clients, far less than that (82%) identified from the progress notes, suggesting the importance of using NLP technology to uncover the information from nursing notes to fully understand the health status of the vulnerable older people in residential aged care. CONCLUSION: This study identified 33% of older people suffered from malnutrition, lower than those reported in the similar setting in previous studies. Our study demonstrates that NLP technology is important for uncovering the key information about health risks for older people in residential aged care. Future research can apply NLP to predict other health risks for older people in this setting.

Agitation Prevalence in People With Dementia in Australian Residential Aged Care Facilities: Findings From Machine Learning of Electronic Health Records.

Using a suite of artificial intelligence technologies, the current study sought to determine the prevalence of agitated behaviors in people with dementia in residential aged care facilities (RACFs) in Australia. Computerized natural language processing allowed extraction of agitation instances from the free-text nursing progress notes, a component of electronic health records in RACFs. In total, 59 observable agitated behaviors were found. No difference was found in dementia prevalence between female and male clients (44.1%), across metropolitan and regional facilities (42.1% [SD = 17.9%]), or for agitation prevalence in dementia (76.5% [SD = 18.4%]). The top 10 behaviors were resisting, wandering, speaking in excessively loud voice, pacing, restlessness, pushing, shouting, complaining, frustration, and using profane language. Four to 17 agitated behaviors coexisted in 53% of people with dementia agitation, indicating high caregiver burden in these RACFs. Improving workforce training and redesigning care models are urgent for sustainability of dementia care in RACFs. [Journal of Gerontological Nursing, 48(4), 57-64.].