Assessing question characteristic influences on ChatGPT's performance and response-explanation consistency: Insights from Taiwan's Nursing Licensing Exam.

BACKGROUND: Investigates the integration of an artificial intelligence tool, specifically ChatGPT, in nursing education, addressing its effectiveness in exam preparation and self-assessment. OBJECTIVE: This study aims to evaluate the performance of ChatGPT, one of the most promising artificial intelligence-driven linguistic understanding tools in answering question banks for nursing licensing examination preparation. It further analyzes question characteristics that might impact the accuracy of ChatGPT-generated answers and examines its reliability through human expert reviews. DESIGN: Cross-sectional survey comparing ChatGPT-generated answers and their explanations. SETTING: 400 questions from Taiwan's 2022 Nursing Licensing Exam. METHODS: The study analyzed 400 questions from five distinct subjects of Taiwan's 2022 Nursing Licensing Exam using the ChatGPT model which provided answers and in-depth explanations for each question. The impact of various question characteristics, such as type and cognitive level, on the accuracy of the ChatGPT-generated responses was assessed using logistic regression analysis. Additionally, human experts evaluated the explanations for each question, comparing them with the ChatGPT-generated answers to determine consistency. RESULTS: ChatGPT exhibited overall accuracy at 80.75 % for Taiwan's National Nursing Exam, which passes the exam. The accuracy of ChatGPT-generated answers diverged significantly across test subjects, demonstrating a hierarchy ranging from General Medicine at 88.75 %, Medical-Surgical Nursing at 80.0 %, Psychology and Community Nursing at 70.0 %, Obstetrics and Gynecology Nursing at 67.5 %, down to Basic Nursing at 63.0 %. ChatGPT had a higher probability of eliciting incorrect responses for questions with certain characteristics, notably those with clinical vignettes [odds ratio 2.19, 95 % confidence interval 1.24-3.87, P = 0.007] and complex multiple-choice questions [odds ratio 2.37, 95 % confidence interval 1.00-5.60, P = 0.049]. Furthermore, 14.25 % of ChatGPT-generated answers were inconsistent with their explanations, leading to a reduction in the overall accuracy to 74 %. CONCLUSIONS: This study reveals the ChatGPT's capabilities and limitations in nursing exam preparation, underscoring its potential as an auxiliary educational tool. It highlights the model's varied performance across different question types and notable inconsistencies between its answers and explanations. The study contributes significantly to the understanding of artificial intelligence in learning environments, guiding the future development of more effective and reliable artificial intelligence-based educational technologies. TWEETABLE ABSTRACT: New study reveals ChatGPT's potential and challenges in nursing education: Achieves 80.75 % accuracy in exam prep but faces hurdles with complex questions and logical consistency. #AIinNursing #AIinEducation #NursingExams #ChatGPT.

Perception of transitional care quality associated with functional outcomes among patients with fractures and stroke in Taiwan.

This study aimed to examine the relationship between self-perceived quality of transitional care and functional outcome among patients with stroke and fractures. The Care Transition Measure (CTM-15) was used to survey patient's self-perceived transitional care quality before discharge. General estimating equations were used to investigate the influences of transitional care quality on patient's functional outcomes at before, 1 week after, and 1 or 3 months after discharge. Among stroke patients, higher CTM-15 scores were positively associated with greater outcome in Instrumental Activities of Daily Living (IADL) following discharge. Higher scores for "reader-friendly written care plan," "consideration of patient's preferences," and "understanding of health management" had significantly positive effects on functional recovery in IADL among both patient groups following discharge. These findings suggest that heterogeneity in transitional care needs between medical and surgical patients shall not be overlooked. A one-size-fits-all strategy may be insufficient for ensuring patient care continuity following discharge.

Effectiveness of a video-based advance care planning intervention in hospitalized elderly patients: A randomized controlled trial.

AIM: This study aimed to develop a video-based advance care planning intervention, and examine the effectiveness of the intervention on knowledge, behavioral intentions and advance directive signing among older patients admitted to a geriatric unit in Taiwan. METHODS: This randomized controlled trial was carried out in two geriatric wards of a medical center in northern Taiwan, between June 2014 and June 2015. The participants were randomly assigned to the intervention (n = 39) or control (n = 43) group. Participants in the intervention group watched a 5-min video regarding advance care planning. The control group received usual care. Structured questionnaires on advance care planning knowledge and behavioral intentions were administered to both groups before and after the intervention. Advance directive signing was carried out until 2 weeks after hospital discharge through a chart review. RESULTS: The intervention group showed a significantly higher advance care planning knowledge score and behavioral intention score than the control group post-intervention. The rate of advance directive signing was 33.3% in the intervention group and 9.3% in the control group (P = 0.01). CONCLUSIONS: This study showed the effectiveness of video-based interventions in increasing advance care planning knowledge, behavioral intention and advance directive signing among hospitalized older patients. Geriatric wards could use our model to promote advance care planning among older patients. Geriatr Gerontol Int 2021; 21: 478-484.

LACE Score-Based Risk Management Tool for Long-Term Home Care Patients: A Proof-of-Concept Study in Taiwan.

BACKGROUND: Effectively predicting and reducing readmission in long-term home care (LTHC) is challenging. We proposed, validated, and evaluated a risk management tool that stratifies LTHC patients by LACE predictive score for readmission risk, which can further help home care providers intervene with individualized preventive plans. METHOD: A before-and-after study was conducted by a LTHC unit in Taiwan. Patients with acute hospitalization within 30 days after discharge in the unit were enrolled as two cohorts (Pre-Implement cohort in 2017 and Post-Implement cohort in 2019). LACE score performance was evaluated by calibration and discrimination (AUC, area under receiver operator characteristic (ROC) curve). The clinical utility was evaluated by negative predictive value (NPV). RESULTS: There were 48 patients with 87 acute hospitalizations in Pre-Implement cohort, and 132 patients with 179 hospitalizations in Post-Implement cohort. These LTHC patients were of older age, mostly intubated, and had more comorbidities. There was a significant reduction in readmission rate by 44.7% (readmission rate 25.3% vs. 14.0% in both cohorts). Although LACE score predictive model still has room for improvement (AUC = 0.598), it showed the potential as a useful screening tool (NPV, 87.9%; 95% C.I., 74.2-94.8). The reduction effect is more pronounced in infection-related readmission. CONCLUSION: As real-world evidence, LACE score-based risk management tool significantly reduced readmission by 44.7% in this LTHC unit. Larger scale studies involving multiple homecare units are needed to assess the generalizability of this study.

Advance Care Planning to Improve End-of-Life Decision-Making Consistency between Older People and Their Surrogates in Taiwan.

Background: Surrogates often do not accurately predict older people's preferences about end-of-life (EOL) care. Few studies have examined the impact of advance care planning (ACP) on EOL decision-making consistency between older people and their surrogates, and these studies have yielded conflicting results. Objectives: To evaluate the effectiveness of ACP in improving EOL decision-making consistency between older people and their surrogates. Design: The intervention in this pre-post quasi-experimental design included an informative video, a brochure about ACP, and a guided discussion about EOL wishes. Setting: Two geriatric wards in a medical center in northern Taiwan. Subjects: One hundred eight participants, as 54 pairs of older people and their surrogates, were randomly assigned to either the experimental or control group. The experimental group received an intervention, while the control group received usual care. Measurements: Life-Support Preferences Questionnaire. Results: The intervention did not improve decision-making consistency between older people and their surrogates. This was the first time that most pairs discussed specific EOL decisions, so additional preparation may improve comfort with this topic. This study also found that some older people had difficulty concentrating on the educational brochure or understanding the related terms. Conclusions: Preparation for ACP discussion is needed for older people and their surrogates. Longer-term effects of ACP should be monitored because ACP interventions may have enhanced empathy between older people and their surrogates. Additionally, a culturally sensitive illustrated questionnaire that explains life-support preferences and ACP topics may improve communication between older people and their surrogates.

Knee joint function, walking ability and quality of life within 6 weeks after total knee arthroplasty: A prospective cohort study.

AIMS AND OBJECTIVES: To track changes in patients' knee function, walking ability and quality of life (QOL) before and after knee arthroplasty surgery and explore factors that influenced these parameters. BACKGROUND: The number of days as an inpatient for those who undergo joint replacement operations has been greatly reduced in recent years. However, it remains unclear whether shortening the inpatient time has affected patients' recovery or their QOL. DESIGN: Prospective cohort study. METHOD: In total, 101 participants who underwent total knee arthroplasty (TKA) were enrolled, with data collected preoperatively and at the 2nd and 6th weeks postoperatively. We followed the STROBE checklist to ensure the rigour of our study. RESULTS: Forty-nine participants underwent unilateral TKA (UTKA; 48.5%), and 52 underwent simultaneous bilateral TKA (SBTKA; 51.5%). At the 2nd week postoperatively, knee joint function was significantly worse than that preoperatively. However, these patients did not exhibit significant differences in the EQ-5D utility index (EQ-5D UI) compared to their preoperative scores. At the 6th week postoperatively, the knee joint function and EQ-5D UI were significantly better than preoperative values. Participants who underwent UTKA scored higher on the Oxford knee score (OKS) than did SBTKA participants. However, no significant differences were noted between the two groups for the knee flexion angle, the timed up and go (TUG) test, or the EQ-5D UI score. CONCLUSIONS AND SUGGESTIONS: At the postoperative 6th week, participants exhibited improved knee joint function and QOL, but there was no difference in the walking ability compared to preoperative values. At the 6th week after surgery, patients who underwent SBTKA exhibited poorer joint function than did those who underwent UTKA. RELEVANCE TO CLINICAL PRACTICE: These results can assist nursing staffs and serve as a reference for providing patient education and nursing intervention to TKA patients.

Using chest vibration nursing intervention to improve expectoration of airway secretions and prevent lung collapse in ventilated ICU patients: a randomized controlled trial.

BACKGROUND: Almost 80% of patients in the intensive care unit are intubated and on mechanical ventilation. Thus, their airway clearance ability is compromised and their risk of lung collapse increased. A variety of interventions are used to enhance airway clearance with the goal of preventing atelectasis and infection. The purpose of this study was to evaluate the effect of a chest vibration nursing intervention on the expectoration of airway secretions and in preventing lung collapse among ventilated critically ill patients. METHODS: This was a randomized, single-blind experimental study. A total of 95 patients were enrolled from 2 ICUs and randomly assigned into either the experimental group (n = 50) or control group (n = 45). Patients in the control group received routine positioning care, which consisted of a change in body position every 2 hours. Patients in the experimental group received routine positioning care plus the use of chest vibration nursing intervention for 72 hours. This intervention consisted of placing a mechanical chest wall vibration pad on the patients back for 60 minutes when the patient was in a supine position. The chest vibration intervention was performed 6 times a day. Outcome variables were dry sputum weight (DSW) per 24 hours and lung collapse index (LCI); these were measured at 24, 48 and 72 hours. RESULTS: Patients who received the chest vibration nursing intervention had greater DSW and lower LCI after 24 hours. Pre-test DSW and group could explain 48.2% of the variance in DSW at 24 hours. The LCI at 24, 48 and 72 hours were all significantly improved in the intervention group compared to the control group. The previous LCI measured was the most significant predictor of the next LCI measured. A significant difference was found between the control and experimental groups in their 24-, 48- and 72-hour DSW and LCI after vibration, when monitored by the generalized estimating equation in time sequence. CONCLUSION: The results suggest that chest vibration may contribute to expectoration and thus improve lung collapse among ventilated patients in an ICU. Chest vibration nursing intervention is a safe and effective alternative pulmonary clearance method and can be used on patients who are on ventilators in ICUs.

The effect of intermittent nasogastric feeding on preventing aspiration pneumonia in ventilated critically ill patients.

This randomized, controlled study employed two feeding protocols for 107 participants in two intensive care units (ICUs) of a medical center to investigate the efficacy of intermittent nasogastric (NG) feeding in preventing aspiration pneumonia in critically ill patients on ventilators. The participants were randomly assigned to receive continuous (51 patients) or intermittent (56 patients) feeding. The primary outcomes, including gastric emptiness index and pulmonary aspiration index, were examined on Day 0 and Day 7 of the intervention. In addition, patients were followed up to the 21st day to evaluate the secondary outcomes, which included length of stay (LOS) in the ICU and airway status. The results showed that the patients in the intermittent feeding group had a higher total intake volume at Day 7 (p = .000), had been extubated earlier at Day 21 (p = .002), and had a lower risk of aspiration pneumonia (odds ratios: 0.146, 95% CI = 0.062-0.413, p = .000) than the patients in the control group. Participants being treated with a high dose of dopamine were 2.95 times more likely to get aspiration pneumonia than those receiving a low dose of dopamine (95 % CI = 1.076-8.107, p = .035). However, there was no significant difference between the two groups'; LOS. The results of this study provide evidence that clinical caregivers may use to make better decisions in terms of feeding methods for critically ill patients.