Improving the Quality of Evaluation Data in Simulation-Based Healthcare Improvement Projects: A Practitioner's Guide to Choosing and Using Published Measurement Tools.

STATEMENT: Simulation is increasingly being used in healthcare improvement projects. The aims of such projects can be extremely diverse. Accordingly, the outcomes or participant attributes that need to be measured can vary dramatically from project-to-project and may include a wide range of nontechnical skills, technical skills, and psychological constructs. Consequently, there is a growing need for simulation practitioners to be able to identify suitable measurement tools and incorporate them into their work. This article provides a practical introduction and guide to the key considerations for practitioners when selecting and using such tools. It also offers a substantial selection of example tools, both to illustrate the key considerations in relation to choosing a measure (including reliability and validity) and to serve as a convenient resource for those planning a study. By making well-informed choices, practitioners can improve the quality of the data they collect, and the likelihood that their projects will succeed.

Interruptions in Health Care: Assessing Their Connection With Error and Patient Harm.

OBJECTIVE: We address the problem of how researchers investigate the actual or potential causal connection between interruptions and medical errors, and whether interventions might reduce the potential for harm. BACKGROUND: It is widely assumed that interruptions lead to errors and patient harm. However, many reviewers and authors have commented that there is not strong evidence for a causal connection. METHOD: We introduce a framework of criteria for assessing how strongly evidence implies causality: the so-called Bradford Hill criteria. We then examine four key "metanarratives" of research into interruptions in health care-applied cognitive psychology, epidemiology, quality improvement, and cognitive systems engineering-and assess how each tradition has addressed the causal connection between interruptions and error. RESULTS: Outcomes of applying the Bradford Hill criteria are that the applied cognitive psychology and epidemiology metanarratives address the causal connection relatively directly, whereas the quality improvement metanarrative merely assumes causality, and the cognitive systems engineering metanarrative either implicitly or explicitly questions the feasibility of finding a direct causal connection with harm. CONCLUSION: The Bradford Hill criteria are useful for evaluating the existing literature on the relationship between interruptions in health care, clinical errors, and the potential for patient harm. In the future, more attention is needed to the issue of why interruptions usually do not lead to harm, and the implications for how we approach patient safety.

Applying social network analysis to the examination of interruptions in healthcare.

Examinations of interruptions in healthcare often focus on a single clinical discipline, and solutions are targeted accordingly. This approach does not take into account the inter-disciplinary dependencies and other sociotechnical aspects that make up the healthcare work system, and suggested solutions may not meet the needs of all stakeholders. In this article a sociotechnical systems perspective is used to uncover the interdependencies between 16 unique work roles that result in interruptions in an intensive care unit (ICU). By applying social network analysis techniques to data collected using the Dual Perspectives Method, we identified targeted systems-based interventions that may reduce unnecessary interruptions while avoiding unintended consequences that impose additional burden on ICU staff. The rich insights gained into the interruptive communication patterns in the ICU work system stand in contrast to findings that would have otherwise been obtained by focusing only on a single clinical discipline or a single perspective.

Deconstructing Cancer Patient Information Seeking in a Consumer Health Library Toward Developing a Virtual Information Consult for Cancer Patients and Their Caregivers: A Qualitative, Instrumental Case Study.

BACKGROUND: Cancer patients and their caregivers want information about their disease and are interested in finding health information online. Despite the abundance of cancer information online, it is often fragmented, its quality is highly variable, and it can be difficult to navigate without expert-level knowledge of the cancer system. The Patient & Family Library at the Princess Margaret Cancer Centre offers a broad collection of high-quality cancer health information and staff are available to help patrons refine their questions and explore information needs that they may not have considered. OBJECTIVE: The purpose of this research study was to deconstruct patrons' information-seeking behaviors in the library to assess the feasibility of replicating the services provided in the library through a Web app, extending the service beyond the walls of the cancer centre. The specific aims of this research were to understand (1) how patrons approach information seeking in the library (interface design), (2) how patrons communicate their informational needs (information categorization and metadata requirements), and (3) what resources are provided to address the patrons' information needs (collection development). METHODS: We employed a qualitative, instrumental case study to deconstruct patrons' health information-seeking behavior. The study population included patients, the librarian, and library volunteers. Ethnographic observation was conducted at the library over 3 days and key informant interviews with library staff were conducted to address the first aim. A closed card-sorting activity was conducted to address the second aim and the library shift logs and Search Request Forms (SRFs) were reviewed to address the third aim. RESULTS: A total of 55 interactions were recorded during the ethnographic observation and nine semistructured interviews were conducted during the key informant interviews. Seven library patron personas were identified: (1) Newbie, (2) Seasoned, (3) Direct, (4) Window Shopper, (5) Collector, (6) Information Seeker, and (7) Distressed. A total of 83 participants completed the closed card-sorting exercise. The participants' conceptual clusters within the similarity matrix overlapped with the groupings created by the librarian, with a few differences. A total of 161 entries in the library shift log and 65 SRFs were analyzed to determine what resources were given to patrons. Most resources that patrons received were available online (61%), although almost half of these required special access (47%). CONCLUSIONS: The study findings suggest it is possible to replicate library functions in a Web app with a few exceptions that cannot be replicated online. These elements include access to journal articles or other content behind paywalls and the librarian's ability to encourage further discussion through empathy and active listening. Discussion with the librarian could serve to refine and predict needs through observing information seekers and to provide immediate connection to spiritual care and psychosocial support for patrons in distress.

Traditions of research into interruptions in healthcare: A conceptual review.

BACKGROUND: Researchers from diverse theoretical backgrounds have studied workplace interruptions in healthcare, leading to a complex and conflicting body of literature. Understanding pre-existing viewpoints may advance the field more effectively than attempts to remove bias from investigations. OBJECTIVE: To identify research traditions that have motivated and guided interruptions research, and to note research questions posed, gaps in approach, and possible avenues for future research. METHODS: A critical review was conducted of research on interruptions in healthcare. Two researchers identified core research communities based on the community's motivations, philosophical outlook, and methods. Among the characteristics used to categorise papers into research communities were the predominant motivation for studying interruptions, the research questions posed, and key contributions to the body of knowledge on interruptions in healthcare. In cases where a paper approached an equal number of characteristics from two traditions, it was placed in a blended research community. RESULTS: A total of 141 papers were identified and categorised; all papers identified were published from 1994 onwards. Four principal research communities emerged: epidemiology, quality improvement, cognitive systems engineering (CSE), and applied cognitive psychology. Blends and areas of mutual influence between the research communities were identified that combine the benefits of individual traditions, but there was a notable lack of blends incorporating quality improvement initiatives. The question most commonly posed by researchers across multiple communities was: what is the impact of interruptions? Impact was measured as a function of task time or risk in the epidemiology tradition, situation awareness in the CSE tradition, or resumption lag (time to resume an interrupted task) in the applied cognitive psychology tradition. No single question about interruptions in healthcare was shared by all four of the core communities. CONCLUSIONS: Much research on workplace interruptions in healthcare can be described in terms of fundamental values of four distinct research traditions and the communities that bring the values and methods: of those research traditions to their investigations. Blends between communities indicate that mutual influence has occurred as interruptions research has progressed. It is clear from this review that there is no single or privileged perspective to study interruptions. Instead, these findings suggest that researchers investigating interruptions in healthcare would benefit from being more aware of different perspectives from their own, especially when they consider workplace interventions to reduce interruptions.

Two sides to every story: The Dual Perspectives Method for examining interruptions in healthcare.

Interruptions are widely considered a problem in healthcare. Results from observation and experimental studies have guided extensive mitigation efforts, but the effectiveness of interventions remains mixed. We have built on current theories and methods for studying interruptions to develop a novel observational approach - the Dual Perspectives Method - for examining interruptions from the perspectives of the different work functions in an Intensive Care Unit (ICU). We detail the method and provide representative examples of the insights it offers, such as why interruptions happen, the role they play, and the consequences of preserving them or eliminating them. We anticipate that the Dual Perspectives Method will help us to arrive at a better basis on which to draw conclusions about interruptions, and will lead to the development of appropriate and sustainable interventions to ensure the effective and safe functioning of the work system under examination.

Mitigating errors caused by interruptions during medication verification and administration: interventions in a simulated ambulatory chemotherapy setting.

BACKGROUND: Nurses are frequently interrupted during medication verification and administration; however, few interventions exist to mitigate resulting errors, and the impact of these interventions on medication safety is poorly understood. OBJECTIVE: The study objectives were to (A) assess the effects of interruptions on medication verification and administration errors, and (B) design and test the effectiveness of targeted interventions at reducing these errors. METHODS: The study focused on medication verification and administration in an ambulatory chemotherapy setting. A simulation laboratory experiment was conducted to determine interruption-related error rates during specific medication verification and administration tasks. Interventions to reduce these errors were developed through a participatory design process, and their error reduction effectiveness was assessed through a postintervention experiment. RESULTS: Significantly more nurses committed medication errors when interrupted than when uninterrupted. With use of interventions when interrupted, significantly fewer nurses made errors in verifying medication volumes contained in syringes (16/18; 89% preintervention error rate vs 11/19; 58% postintervention error rate; p=0.038; Fisher's exact test) and programmed in ambulatory pumps (17/18; 94% preintervention vs 11/19; 58% postintervention; p=0.012). The rate of error commission significantly decreased with use of interventions when interrupted during intravenous push (16/18; 89% preintervention vs 6/19; 32% postintervention; p=0.017) and pump programming (7/18; 39% preintervention vs 1/19; 5% postintervention; p=0.017). No statistically significant differences were observed for other medication verification tasks. CONCLUSIONS: Interruptions can lead to medication verification and administration errors. Interventions were highly effective at reducing unanticipated errors of commission in medication administration tasks, but showed mixed effectiveness at reducing predictable errors of detection in medication verification tasks. These findings can be generalised and adapted to mitigate interruption-related errors in other settings where medication verification and administration are required.