Developing and validating a knowledge-based AI assessment system for learning clinical core medical knowledge in otolaryngology.

BACKGROUND: Clinical core medical knowledge (CCMK) learning is essential for medical trainees. Adaptive assessment systems can facilitate self-learning, but extracting experts' CCMK is challenging, especially using modern data-driven artificial intelligence (AI) approaches (e.g., deep learning). OBJECTIVES: This study aims to develop a multi-expert knowledge-aggregated adaptive assessment scheme (MEKAS) using knowledge-based AI approaches to facilitate the learning of CCMK in otolaryngology (CCMK-OTO) and validate its effectiveness through a one-month training program for CCMK-OTO education at a tertiary referral hospital. METHODS: The MEKAS utilized the repertory grid technique and case-based reasoning to aggregate experts' knowledge to construct a representative CCMK base, thereby enabling adaptive assessment for CCMK-OTO training. The effects of longitudinal training were compared between the experimental group (EG) and the control group (CG). Both groups received a normal training program (routine meeting, outpatient/operation room teaching, and classroom teaching), while EG received MEKAS for self-learning. The EG comprised 22 UPGY trainees (6 postgraduate [PGY] and 16 undergraduate [UGY] trainees) and 8 otolaryngology residents (ENT-R); the CG comprised 24 UPGY trainees (8 PGY and 16 UGY trainees). The training effectiveness was compared through pre- and post-test CCMK-OTO scores, and user experiences were evaluated using a technology acceptance model-based questionnaire. RESULTS: Both UPGY (z = -3.976, P < 0.001) and ENT-R (z = -2.038, P = 0.042) groups in EG exhibited significant improvements in their CCMK-OTO scores, while UPGY in CG did not (z = -1.204, P = 0.228). The UPGY group in EG also demonstrated a substantial improvement compared to the UPGY group in CG (z = -4.943, P < 0.001). The EG participants were highly satisfied with the MEKAS system concerning self-learning assistance, adaptive testing, perceived satisfaction, intention to use, perceived usefulness, perceived ease of use, and perceived enjoyment, rating it between an overall average of 3.8 and 4.1 out of 5.0 on all scales. CONCLUSIONS: The MEKAS system facilitates CCMK-OTO learning and provides an efficient knowledge aggregation scheme that can be applied to other medical subjects to efficiently build adaptive assessment systems for CCMK learning. Larger-scale validation across diverse institutions and settings is warranted further to assess MEKAS's scalability, generalizability, and long-term impact.

A scenario-based web app to facilitate patient education in lung tumor patients undergoing video-assisted thoracoscopic surgery: Development and usability testing.

Background: Patient education (PE) is essential for improving patients' knowledge, anxiety, and satisfaction, and supporting their postoperative recovery. However, the advantages of video-assisted thoracoscopic surgery (VATS)-smaller incisions and faster recovery-can result in shorter hospital stays, making PE more challenging to implement effectively. Multimedia PE can potentially enhance PE, but its effectiveness for patients undergoing VATS is unclear. Objective: This study developed a scenario-based PE web app for lung tumor patients undergoing VATS (SPE-VATS) to facilitate the PE process and evaluated its usability through a clinical trial. Methods: The SPE-VATS provided the experimental group (EG: 32 participants) with interactive scenario, query guidance, diagnostic analysis, experience sharing, and active reminder, while the control group (CG: 32 participants) used pamphlets and videos. The usability of SPE-VATS in terms of postoperative anxiety reduction and patient satisfaction with the app was evaluated using self-reported questionnaires based on the state-trait anxiety inventory, technology acceptance model, system usability scale, and task load index. Results: There was no statistically significant difference in postoperative anxiety reduction between the EG and CG, possibly because 90% of the participants underwent a low-risk surgical type, and VATS is known to be advantageous in alleviating surgical anxiety. However, females and higher educated EG participants showed a non-significant but favorable reduction than their CG counterparts. Moreover, the EG was highly satisfied with the app (rated 4.2 to 4.4 out of 5.0), with no significant gender and education level difference. They particularly valued the interactive scenario, experience sharing, and diagnostic analysis features of SPE-VATS. Conclusions: The SPE-VATS demonstrated its usability and high patient satisfaction, particularly for female and higher educated patients. Low-risk patient predominance and VATS's advantages may explain non-significant postoperative anxiety reduction, warranting further studies on high-risk patients to evaluate the impact of SPE-VATS on clinical practice.

Machine learning approach to determine the decision rules in ergonomic assessment of working posture in sewing machine operators.

INTRODUCTION: There are some inherent problems with the use of observation methods in the ergonomic assessment of working posture, namely the stability and precision of the measurements. This study aims to use a machine learning (ML) approach to avoid the subjectivity bias of observational methods in ergonomic assessments and further identify risk patterns for work-related musculoskeletal disorders (WMSDs) among sewing machine operators. METHODS: We proposed a decision tree analysis scheme for ergonomic assessment in working postures (DTAS-EAWP). First, DTAS-EAWP used computer vision-based technology to detect the body movement angles from the on-site working videos to generate a dataset of risk scores through the criteria of Rapid Entire Body Assessment (REBA) for sewing machine operators. Second, data mining techniques (WEKA) using the C4.5 algorithm were used to construct a representative decision tree (RDT) with paths of various risk levels, and attribute importance analysis was performed to determine the critical body segments for WMSDs. RESULTS: DTAS-EAWP was able to recognize 11,211 samples of continuous working postures in sewing machine operation and calculate the corresponding final REBA scores. A total of 13 decision rules were constructed in the RDT, with over 95% prediction accuracy and 83% path coverage, to depict the possible risk tendency in the working postures. Through RDT and attribute importance analysis, it was identified that the lower arm and the upper arms exhibited as critical segments that significantly increased the risk levels for WMSDs. CONCLUSIONS: This study demonstrates that ML approach with computer vision-based estimation and DT analysis are feasible for comprehensively exploring the decision rules in ergonomic assessment of working postures for risk prediction of WMSDs in sewing machine operators. PRACTICAL APPLICATIONS: This DTAS-EAWP can be applied in manufacturing industries to automatically analyze working postures and identify risk patterns of WMSDs, leading to the development of effectively preventive interventions.

A mobile-based airway clearance care system using deep learning-based vision technology to support personalized home-based pulmonary rehabilitation for COAD patients: Development and usability testing.

Background: Excessive mucus secretion is a serious issue for patients with chronic obstructive airway disease (COAD), which can be effectively managed through postural drainage and percussion (PD + P) during pulmonary rehabilitation (PR). Home-based (H)-PR can be as effective as center-based PR but lacks professional supervision and timely feedback, leading to low motivation and adherence. Telehealth home-based pulmonary (TH-PR) has emerged to assist H-PR, but video conferencing and telephone calls remain the main approaches for COAD patients. Therefore, research on effectively assisting patients in performing PD + P during TH-PR is limited. Objective: This study developed a mobile-based airway clearance care for chronic obstructive airway disease (COAD-MoAcCare) system to support personalized TH-PR for COAD patients and evaluated its usability through expert validation. Methods: The COAD-MoAcCare system uses a mobile device through deep learning-based vision technology to monitor, guide, and evaluate COAD patients' PD + P operations in real time during TH-PR programs. Medical personnel can manage and monitor their personalized PD + P and operational statuses through the system to improve TH-PR performance. Respiratory therapists from different hospitals evaluated the system usability using system questionnaires based on the technology acceptance model, system usability scale (SUS), and task load index (NASA-TLX). Results: Eleven participant therapists were highly satisfied with the COAD-MoAcCare system, rating it between 4.1 and 4.6 out of 5.0 on all scales. The system demonstrated good usability (SUS score of 74.1 out of 100) and a lower task load (NASA-TLX score of 30.0 out of 100). The overall accuracy of PD + P operations reached a high level of 97.5% by comparing evaluation results of the system by experts. Conclusions: The COAD-MoAcCare system is the first mobile-based method to assist COAD patients in conducting PD + P in TH-PR. It was proven to be usable by respiratory therapists, so it is expected to benefit medical personnel and COAD patients. It will be further evaluated through clinical trials.

Application of mobile-based web app to enhance simple suturing skills of nurse practitioners.

BACKGROUND: Suturing is a crucial clinical skill for nurse practitioners (NPs), but the effectiveness of traditional training methods (e.g., physical suture kits combined with video content) is low. OBJECTIVE: This study compared the effectiveness and usability of a mobile-based web app (MoWa) developed for NPs to learn simple suturing skills with those of traditional instructional video-based training. METHODS: The MoWa system utilizes mobile devices to simulate hands-on suturing and provides learning guidance and feedback to support self-learning with a physical suturing kit. Fifty-four suturing novices (NPs) were recruited as participants, divided into an experimental group (EG: 28 participants) and a control group (CG: 26 participants), and instructed to self-learn for 3 weeks. Learning effectiveness and system usability were evaluated through a pretest and posttest. RESULTS: The EG exhibited significant improvements in learning outcomes, self-confidence, self-efficacy, and learning anxiety and expressed satisfaction with the MoWa system. Furthermore, the EG also considerably enhanced learning outcomes, self-efficacy, and learning anxiety compared to the CG, with no significant difference in self-confidence. CONCLUSION: The MoWa system combined with deliberate practice is an effective strategy for supporting suturing skills training.

Development of a mobile tele-education system to assist remote otolaryngology learning during COVID-19 pandemic.

Background: Developing clinical thinking competence (CTC) is crucial for physicians, but effective methods for cultivation and evaluation are a significant challenge. Classroom teaching and paper-and-pencil tests are insufficient, and clinical field learning is difficult to implement, especially during the COVID-19 pandemic. Simulation learning is a useful alternative, but existing methods, e.g., OSCE, 3D AR/VR, and SimMan, have limitations in terms of time, space, and cost. Objective: This study aims to present the design and development of an Otolaryngology Mobile Tele-education System (OMTS) to facilitate CTC learning, and to evaluate the system's usability with senior otolaryngology experts. Methods: The OMTS system utilizes the convenience of mobile learning and the touch function of mobile devices to assist users (medical students or post-graduate physicians) in learning CTC remotely. Clinical knowledge and system functions in the OMTS system are defined by senior experts based on required CTC learning cases. Through simulated clinical case scenarios, users can engage in interactive clinical inquiry, practice required physical and laboratory examinations, make treatment decisions based on simulated responses, and understand and correct learning problems through a diagnostic report for effective learning. Usability testing of the OMTS system was evaluated by three senior otolaryngology experts using measurements of content validity, system usability, and mental workload during their available time and location. Results: Statistical results of experts' evaluation showed that the OMTS system has good content validity, marginal-to-acceptable system usability, and moderate mental workload. Experts agreed that the system was efficient, professional, and usable for learning, although the practicality of the clinical inquiry and hands-on practice functions could be improved further. Conclusions: Based on the OMTS system, users can efficiently hands-on practice and learn clinical cases in otolaryngology, and understand and correct their problems according to the diagnostic report. Therefore, the OMTS system can be expected to facilitate CTC learning according to experts' evaluation.