Assessing question characteristic influences on ChatGPT's performance and response-explanation consistency: Insights from Taiwan's Nursing Licensing Exam.

BACKGROUND: Investigates the integration of an artificial intelligence tool, specifically ChatGPT, in nursing education, addressing its effectiveness in exam preparation and self-assessment. OBJECTIVE: This study aims to evaluate the performance of ChatGPT, one of the most promising artificial intelligence-driven linguistic understanding tools in answering question banks for nursing licensing examination preparation. It further analyzes question characteristics that might impact the accuracy of ChatGPT-generated answers and examines its reliability through human expert reviews. DESIGN: Cross-sectional survey comparing ChatGPT-generated answers and their explanations. SETTING: 400 questions from Taiwan's 2022 Nursing Licensing Exam. METHODS: The study analyzed 400 questions from five distinct subjects of Taiwan's 2022 Nursing Licensing Exam using the ChatGPT model which provided answers and in-depth explanations for each question. The impact of various question characteristics, such as type and cognitive level, on the accuracy of the ChatGPT-generated responses was assessed using logistic regression analysis. Additionally, human experts evaluated the explanations for each question, comparing them with the ChatGPT-generated answers to determine consistency. RESULTS: ChatGPT exhibited overall accuracy at 80.75 % for Taiwan's National Nursing Exam, which passes the exam. The accuracy of ChatGPT-generated answers diverged significantly across test subjects, demonstrating a hierarchy ranging from General Medicine at 88.75 %, Medical-Surgical Nursing at 80.0 %, Psychology and Community Nursing at 70.0 %, Obstetrics and Gynecology Nursing at 67.5 %, down to Basic Nursing at 63.0 %. ChatGPT had a higher probability of eliciting incorrect responses for questions with certain characteristics, notably those with clinical vignettes [odds ratio 2.19, 95 % confidence interval 1.24-3.87, P = 0.007] and complex multiple-choice questions [odds ratio 2.37, 95 % confidence interval 1.00-5.60, P = 0.049]. Furthermore, 14.25 % of ChatGPT-generated answers were inconsistent with their explanations, leading to a reduction in the overall accuracy to 74 %. CONCLUSIONS: This study reveals the ChatGPT's capabilities and limitations in nursing exam preparation, underscoring its potential as an auxiliary educational tool. It highlights the model's varied performance across different question types and notable inconsistencies between its answers and explanations. The study contributes significantly to the understanding of artificial intelligence in learning environments, guiding the future development of more effective and reliable artificial intelligence-based educational technologies. TWEETABLE ABSTRACT: New study reveals ChatGPT's potential and challenges in nursing education: Achieves 80.75 % accuracy in exam prep but faces hurdles with complex questions and logical consistency. #AIinNursing #AIinEducation #NursingExams #ChatGPT.

Risk Factors and Incidence Rates of Self-Reported Short-Term Adverse Events of COVID-19 Vaccine Booster Dose.

With the spread of the new SARS-CoV-2 variants, many countries have begun COVID-19 vaccine booster programs with the mix-and-match strategy. However, research on the adverse events (AE) of booster doses is still scarce. The aim of our study was to analyze the reported incidence rate (IR), and factors associated with AE, including short-term serious adverse events (SAE) and short-term non-serious adverse events (NSAE), among different vaccine products through the hospital-based Vaccine Adverse Event Reporting System (VAERS). A total of 7432 records were collected during the three-month study period. While more than half of the responses (52.2%) reported the presence of AE after receiving a booster dose, only a few AE were considered SAE (2.4%). AE were significantly higher among women and people of younger age, and the brand of vaccines is the strongest factor associated with post-booster dose AE. The incidence of AE in mRNA1273 is higher than in BNT162b2 and MVC-COV1901 (IRR mRNA1273 vs. BNT162b2: 1.22, 95% CI: 1.11-1.34; BNT162b2 vs. MVC-COV1901: 2.77, 95% CI: 2.27-3.39). The IR of different groups were calculated to support the decision making of the booster vaccine. Although AE were not uncommon for booster vaccines, almost all AE were not serious and predictable using estimated IR. This result can be used to optimize booster vaccine decision making.

Personalized and Self-Management: Systematic Search and Evaluation Quality Factors and User Preference of Drug Reference Apps in Taiwan.

BACKGROUND: Drug reference apps promote self-management and improve the efficiency and quality of work for physicians, nurses, pharmacists, and patients. This study aimed to describe a systematic and stepwise process to identify drug reference apps in Taiwan, assess the quality of these apps, and analyze the influential factors for user ratings. METHODS: A two-step algorithm (KESS) consisting of keyword growing and systematic search was proposed. Seven independent reviewers were trained to evaluate these apps using Mobile App Rating Scale (MARS). A logistic regression model was fitted and average marginal effects (AME) were calculated to identify the effects of factors for higher user ratings. RESULTS: A total of 23 drug reference apps in Taiwan were identified and analyzed. Generally, these drug reference apps were evaluated as acceptable quality with an average MARS score of 3.23. Higher user engagement, more functionality, better aesthetics, and more information associated with higher user ratings. Navigation is the most influential factor on higher user ratings (AME: 13.15%) followed by performance (AME: 11.03%), visual appeal (AME: 10.87%), credibility (AME: 10.67%), and quantity of information (AME: 10.42%). CONCLUSIONS: User experience and information clearly affect user ratings of drug reference apps. Five key factors should be considered when designing drug reference apps.

LACE Score-Based Risk Management Tool for Long-Term Home Care Patients: A Proof-of-Concept Study in Taiwan.

BACKGROUND: Effectively predicting and reducing readmission in long-term home care (LTHC) is challenging. We proposed, validated, and evaluated a risk management tool that stratifies LTHC patients by LACE predictive score for readmission risk, which can further help home care providers intervene with individualized preventive plans. METHOD: A before-and-after study was conducted by a LTHC unit in Taiwan. Patients with acute hospitalization within 30 days after discharge in the unit were enrolled as two cohorts (Pre-Implement cohort in 2017 and Post-Implement cohort in 2019). LACE score performance was evaluated by calibration and discrimination (AUC, area under receiver operator characteristic (ROC) curve). The clinical utility was evaluated by negative predictive value (NPV). RESULTS: There were 48 patients with 87 acute hospitalizations in Pre-Implement cohort, and 132 patients with 179 hospitalizations in Post-Implement cohort. These LTHC patients were of older age, mostly intubated, and had more comorbidities. There was a significant reduction in readmission rate by 44.7% (readmission rate 25.3% vs. 14.0% in both cohorts). Although LACE score predictive model still has room for improvement (AUC = 0.598), it showed the potential as a useful screening tool (NPV, 87.9%; 95% C.I., 74.2-94.8). The reduction effect is more pronounced in infection-related readmission. CONCLUSION: As real-world evidence, LACE score-based risk management tool significantly reduced readmission by 44.7% in this LTHC unit. Larger scale studies involving multiple homecare units are needed to assess the generalizability of this study.

Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan.

The LACE index and HOSPITAL score models are the two most commonly used prediction models identifying patients at high risk of readmission with limited information for home care patients. This study compares the effectiveness of these two models in predicting 30-day readmission following acute hospitalization of such patients in Taiwan. A cohort of 57 home care patients were enrolled and followed-up for one year. We compared calibration, discrimination (area under the receiver operating curve, AUC), and net reclassification improvement (NRI) to identify patients at risk of 30-day readmission for both models. Moreover, the cost-effectiveness of the models was evaluated using microsimulation analysis. A total of 22 readmissions occurred after 87 acute hospitalizations during the study period (readmission rate = 25.2%). While the LACE score had poor discrimination (AUC = 0.598, 95% confidence interval (CI) = 0.488-0.702), the HOSPITAL score achieved helpful discrimination (AUC = 0.691, 95% CI = 0.582-0.785). Moreover, the HOSPITAL score had improved the risk prediction in 38.3% of the patients, compared with the LACE index (NRI = 0.383, 95% CI = 0.068-0.697, p = 0.017). Both prediction models effectively reduced readmission rates compared to an attending physician's model (readmission rate reduction: LACE, 39.2%; HOSPITAL, 43.4%; physician, 10.1%; p < 0.001). The HOSPITAL score provides a better prediction of readmission and has potential as a risk management tool for home care patients.

Severely sustained vomiting as the main symptom in a man with thyrotoxicosis.

Thyrotoxicosis has a variety of presentations. Vomiting as a main presenting symptom of thyrotoxicosis is uncommon. We report a 40-year-old male with thyrotoxicosis who presented with sustained vomiting as the main symptom. He also had weight loss, about 10 kg over this 20-day period, and dizziness, particularly in the upright position. Esophagogastroduodenoscopy and abdominal ultrasonography were negative. Laboratory data were unremarkable except serum T4 of 21.2 microg/dl, T3 of 574 ng/dl and TSH < 0.03 microIU/ml. The patient's serum microsomal antibody was positive at a titer of 1:409,600, but serum thyroglogulin antibody was negative at a titer of less than 1:100. The symptoms improved after adminstroction of propylthyrouracil and propranol. A total of 31 such cases have been reported in English literature. The mean age of the patients was 46 +/- 14 year with a range of 19 to 68 years. Only 4 patients, including ours, were male. Weight loss was found in about half of them and might be an important clue. Thyrotoxicosis should be considered in differential diagnosis of unexplained vomiting.

Chronic glue sniffing with transient central hypothyroidism and hypergonadotropism.

Neuropsychiatric, gastrointestinal and muscular disorders associated with glue sniffing have been widely reported, but endocrinologic abnormalities of glue exposure are rarely mentioned in the literature. We report a 26-year old male patient, a chronic glue sniffer, who presented with weakness of both lower limbs. On physical examination, he had reduced muscle strength of his 4 limbs, especially in his lower limbs. Laboratory examination revealed hypokalemia with hyperchloremic metabolic acidosis. His thyroid function showed low TSH, T4, T3, free T4 and reverse T3 level. Other pituitary functions were normal apart from high FSH and LH level. TSH response to TRH stimulation was normal, but there was impaired T3 response to TRH. MRI of pituitary showed no significant changes. He continued glue sniffing after discharge. He repeatedly came to our hospital for recurrent hypokalemic paralysis. His serum T4 and free T4 level were low when he had certain amount of glue sniffing and it returned to normal after he stopped sniffing or sniffed less amount of glue. His serum T3 concentrations were normal most of the times thereafter. His FSH and LH level were persistently elevated, even after he did not sniff glue for 2 weeks. Low free T4, TSH and reverse T3 level associated with glue sniffing in our patient were compatible with central hypothyroidism. Toluene, a neurotoxic organic solvent, is present in glues. Being highly lipophilic, it can easily enter and is retained within the lipid-rich nervous system after being inhaled. Like other organic solvents, toluene has been shown to affect dopaminergic and adrenergic turnover within various parts of the brain. The effects on these neurotransmitters could lead to abnormal secretion of pituitary hormones resulting in transient central hypothyroidism and abnormal gonadotropin levels. Long-term harmful effect of central hypothyroidism and chronic influence of abnormal gonadotropins to reproduction function needs further observation.