Development of a scoring system to quantify errors from semantic characteristics in incident reports.

OBJECTIVES: Incident reporting systems are widely used to identify risks and enable organisational learning. Free-text descriptions contain important information about factors associated with incidents. This study aimed to develop error scores by extracting information about the presence of error factors in incidents using an original decision-making model that partly relies on natural language processing techniques. METHODS: We retrospectively analysed free-text data from reports of incidents between January 2012 and December 2022 from Nagoya University Hospital, Japan. The sample data were randomly allocated to equal-sized training and validation datasets. We conducted morphological analysis on free text to segment terms from sentences in the training dataset. We calculated error scores for terms, individual reports and reports from staff groups according to report volume size and compared these with conventional classifications by patient safety experts. We also calculated accuracy, recall, precision and F-score values from the proposed 'report error score'. RESULTS: Overall, 114 013 reports were included. We calculated 36 131 'term error scores' from the 57 006 reports in the training dataset. There was a significant difference in error scores between reports of incidents categorised by experts as arising from errors (p<0.001, d=0.73 (large)) and other incidents. The accuracy, recall, precision and F-score values were 0.8, 0.82, 0.85 and 0.84, respectively. Group error scores were positively associated with expert ratings (correlation coefficient, 0.66; 95% CI 0.54 to 0.75, p<0.001) for all departments. CONCLUSION: Our error scoring system could provide insights to improve patient safety using aggregated incident report data.

Development of a Novel Scoring System to Quantify the Severity of Incident Reports: An Exploratory Research Study.

Incident reporting systems have been widely adopted to collect information about patient safety incidents. Much of the value of incident reports lies in the free-text section. Computer processing of semantic information may be helpful to analyze this. We developed a novel scoring system for decision making to assess the severity of incidents using the semantic characteristics of the text in incident reports, and compared its results with experts' opinions. We retrospectively analyzed free-text data from incident reports from January 2012 to September 2021 at Nagoya University Hospital, Aichi, Japan. The sample was allocated to training and validation datasets using the hold-out method. Morphological analysis was used to segment terms in the training dataset. We calculated a severity term score, a severity report score and severity group score, by report volume size, and compared these with conventional severity classifications by patient safety experts and reporters. We allocated 96,082 incident reports into two groups. We calculated 1,802 severity term scores from the 48,041 reports in the training dataset. There was a significant difference in severity report score between reports categorized as severe and not severe by experts (95% confidence interval [CI] -0.83 to -0.80, p < 0.001, d = 0.81). Severity group scores were positively associated with severity ratings from experts and reporters (correlation coefficients 0.73 [95% CI 0.63-0.80, p < 0.001] and 0.79 [95% CI 0.71-0.85, p < 0.001]) for all departments. Our severity scoring system could therefore contribute to better organizational patient safety.

Enhanced hospital-wide communication and interaction by team training to improve patient safety.

Communication errors are the most important cause of adverse events in healthcare. The current study aimed to improve hospital-wide employee teamwork and reduce adverse medical events for patients arising from miscommunication. In our hospital, when patient safety incidents and accidents occur, staff from various occupations submit incident reports to the Department of Patient Safety via an electronic reporting system; over 11,000 cases are reported each year. We surveyed the incident reports submitted in our institution from 2016 to 2018. All incidents related to miscommunication were identified, and relevant information was collected from the original electronic incident reports. Incident severity classification is commonly divided into near-miss or adverse events. We extracted only the required incident information items for this study, and processed information concerning individuals (e.g., reporters and target patients) anonymously. This study was approved by the Institutional Review Board of the study hospital. The authors declare no conflicts of interest associated with this study. Team training for all employees reduced adverse events for patients. The coefficient of determination (R squared value) was -0.32. This suggests our approach may be slightly but significantly effective for developing the fundamental strengths of the medical team. Quality improvement is continuous, and seamless efforts to improve the effectiveness of medical teams at our hospital will continue.

Intervention efficacy for eliminating patient misidentification using step-by-step problem-solving procedures to improve patient safety.

This study aimed to evaluate the efficacy of interventions to reduce patient misidentification incidents classified as level 2 and over (adverse events occurred for patients) with the step-by-step problem-solving method. All incidents related to patient misidentification were selected, and relevant information was collected from the original electronic incident reports. We then conducted an eight-step problem-solving process with the aim of reducing patient misclassification and improving patient safety. Step 1: the number of misidentification-related incident reports and the percentage of these reports in the total incident reports increased each year. Step 2: the most frequent misidentification type was sample collection tubes, followed by drug administration and hospital meals. Step 3: we set a target of an 20% decrease in patient misidentification cases classified as level 2 or over compared with the previous year, and established this as a hospital priority. Step 4: we found that discrepancies in patient identification procedures were the most important causes of misidentification. Step 5: we standardized the patient identification process to achieve an 10% reduction in misidentification. Step 6: we disseminated instructional videos to all staff members. Step 7: we confirmed there was an 18% reduction in level 2 and over patient misidentification compared with the previous year. Step 8: we intend to make additional effort to decrease misidentification of patients by a further 10%. Level 2 and over patient misidentification can be reduced by a patient identification policy using a step-by-step problem-solving procedure. This study aimed to evaluate the efficacy of interventions to reduce patient misidentification incidents with step-by-step problem-solving method. Continued seamless efforts to eliminate patient misidentification are mandatory for this activity.

Significance of incident reports by medical doctors for organizational transparency and driving forces for patient safety.

BACKGROUND: Incident reporting is an effective strategy used to enhance patient safety and quality improvement in healthcare. An incident is an event that could eventually result in harm to a patient. The aim of this study is to re-evaluate the importance of reporting by medical doctors to improve quality in healthcare and patient safety. METHODS: We conducted a retrospective analysis of the reported incidents registered in our institutional database from April 1st 2015 to March 31st 2019, classified according to eight variables proposed by the National University Hospital Council of Japan, to determine the type of incidents and their potential harm to patients. RESULTS: Registered reports totalled 43,775, approximately 8% of which arise annually from medical doctors in clinical departments. Incidents with higher impact on patients have significantly increased the rate of reporting by medical doctors. The most frequent types of report overall concerned medication incidents, followed by infusion lines, drainage-tube devices, cure, examination, and treatment outside the operating room. The most frequent reports by medical doctors involved operation-related incidents, followed by cure, examination, treatment outside the operation room, and medications. CONCLUSION: Reporting by medical doctors reflects the organizational transparency and the driving forces behind patient safety and quality improvement in healthcare. Efforts toward seamless improvement in patient safety and quality at our hospital continue apace.