Developing a sampling method and preliminary taxonomy for classifying COVID-19 public health guidance for healthcare organizations and the general public.

BACKGROUND: Development and dissemination of public health (PH) guidance to healthcare organizations and the general public (e.g., businesses, schools, individuals) during emergencies like the COVID-19 pandemic is vital for policy, clinical, and public decision-making. Yet, the rapidly evolving nature of these events poses significant challenges for guidance development and dissemination strategies predicated on well-understood concepts and clearly defined access and distribution pathways. Taxonomies are an important but underutilized tool for guidance authoring, dissemination and updating in such dynamic scenarios. OBJECTIVE: To design a rapid, semi-automated method for sampling and developing a PH guidance taxonomy using widely available Web crawling tools and streamlined manual content analysis. METHODS: Iterative samples of guidance documents were taken from four state PH agency websites, the US Center for Disease Control and Prevention, and the World Health Organization. Documents were used to derive and refine a preliminary taxonomy of COVID-19 PH guidance via content analysis. RESULTS: Eight iterations of guidance document sampling and taxonomy revisions were performed, with a final corpus of 226 documents. The preliminary taxonomy contains 110 branches distributed between three major domains: stakeholders (24 branches), settings (25 branches) and topics (61 branches). Thematic saturation measures indicated rapid saturation (≤5% change) for the domains of "stakeholders" and "settings", and "topic"-related branches for clinical decision-making. Branches related to business reopening and economic consequences remained dynamic throughout sampling iterations. CONCLUSION: The PH guidance taxonomy can support public health agencies by aligning guidance development with curation and indexing strategies; supporting targeted dissemination; increasing the speed of updates; and enhancing public-facing guidance repositories and information retrieval tools. Taxonomies are essential to support knowledge management activities during rapidly evolving scenarios such as disease outbreaks and natural disasters.

The Value of Monitoring Clinical Decision Support Interventions.

BACKGROUND: Well-functioning clinical decision support (CDS) can facilitate provider workflow, improve patient care, promote better outcomes, and reduce costs. However, poorly functioning CDS may lead to alert fatigue, cause providers to ignore important CDS interventions, and increase provider dissatisfaction. OBJECTIVE: The purpose of this article is to describe one institution's experience in implementing a program to create and maintain properly functioning CDS by systematically monitoring CDS firing rates and patterns. METHODS: Four types of CDS monitoring activities were implemented as part of the CDS lifecycle. One type of monitoring occurs prior to releasing active CDS, while the other types occur at different points after CDS activation. RESULTS: Two hundred and forty-eight CDS interventions were monitored over a 2-year period. The rate of detecting a malfunction or significant opportunity for improvement was 37% during preactivation and 18% during immediate postactivation monitoring. Monitoring also informed the process of responding to user feedback about alerts. Finally, an automated alert detection tool identified 128 instances of alert pattern change over the same period. A subset of cases was evaluated by knowledge engineers to identify true and false positives, the results of which were used to optimize the tool's pattern detection algorithms. CONCLUSION: CDS monitoring can identify malfunctions and/or significant improvement opportunities even after careful design and robust testing. CDS monitoring provides information when responding to user feedback. Ongoing, continuous, and automated monitoring can detect malfunctions in real time, before users report problems. Therefore, CDS monitoring should be part of any systematic program of implementing and maintaining CDS.

Modeling Contextual Knowledge for Clinical Decision Support.

In theory, the logic of decision rules should be atomic. In practice, this is not always possible; initially simple logic statements tend to be overloaded with additional conditions restricting the scope of such rules. By doing so, the original logic soon becomes encumbered with contextual knowledge. Contextual knowledge is re-usable on its own and could be modeled separately from the logic of a rule without losing the intended functionality. We model constraints to explicitly define the context where knowledge of decision rules is actionable. We borrowed concepts from Semantic Web, Complex Adaptive Systems, and Contextual Reasoning. The proposed approach provides the means for identifying and modeling contextual knowledge in a simple, sound manner. The methodology presented herein facilitates rule authoring, fosters consistency in rules implementation and maintenance; facilitates developing authoritative knowledge repositories to promote quality, safety and efficacy of healthcare; and paves the road for future work in knowledge discovery.

A Practical Approach to Governance and Optimization of Structured Data Elements.

Definition and configuration of clinical content in an enterprise-wide electronic health record (EHR) implementation is highly complex. Sharing of data definitions across applications within an EHR implementation project may be constrained by practical limitations, including time, tools, and expertise. However, maintaining rigor in an approach to data governance is important for sustainability and consistency. With this understanding, we have defined a practical approach for governance of structured data elements to optimize data definitions given limited resources. This approach includes a 10 step process: 1) identification of clinical topics, 2) creation of draft reference models for clinical topics, 3) scoring of downstream data needs for clinical topics, 4) prioritization of clinical topics, 5) validation of reference models for clinical topics, and 6) calculation of gap analyses of EHR compared against reference model, 7) communication of validated reference models across project members, 8) requested revisions to EHR based on gap analysis, 9) evaluation of usage of reference models across project, and 10) Monitoring for new evidence requiring revisions to reference model.

Modeling decision support rule interactions in a clinical setting.

Traditionally, rule interactions are handled at implementation time through rule task properties that control the order in which rules are executed. By doing so, knowledge about the behavior and interactions of decision rules is not captured at modeling time. We argue that this is important knowledge that should be integrated in the modeling phase. In this project, we build upon current work on a conceptual schema to represent clinical knowledge for decision support in the form of if then rules. This schema currently captures provenance of the clinical content, context where such content is actionable (i.e. constraints) and the logic of the rule itself. For this project, we borrowed concepts from both the Semantic Web (i.e., Ontologies) and Complex Adaptive Systems (CAS), to explore a conceptual approach for modeling rule interactions in an enterprise-wide clinical setting. We expect that a more comprehensive modeling will facilitate knowledge authoring, editing and update; foster consistency in rules implementation and maintenance; and develop authoritative knowledge repositories to promote quality, safety and efficacy of healthcare.

Structured representation for core elements of common clinical decision support interventions to facilitate knowledge sharing.

At present, there are no widely accepted, standard approaches for representing computer-based clinical decision support (CDS) intervention types and their structural components. This study aimed to identify key requirements for the representation of five widely utilized CDS intervention types: alerts and reminders, order sets, infobuttons, documentation templates/forms, and relevant data presentation. An XML schema was proposed for representing these interventions and their core structural elements (e.g., general metadata, applicable clinical scenarios, CDS inputs, CDS outputs, and CDS logic) in a shareable manner. The schema was validated by building CDS artifacts for 22 different interventions, targeted toward guidelines and clinical conditions called for in the 2011 Meaningful Use criteria. Custom style sheets were developed to render the XML files in human-readable form. The CDS knowledge artifacts were shared via a public web portal. Our experience also identifies gaps in existing standards and informs future development of standards for CDS knowledge representation and sharing.

Practical choices for infobutton customization: experience from four sites.

Context-aware links between electronic health records (EHRs) and online knowledge resources, commonly called "infobuttons" are being used increasingly as part of EHR "meaningful use" requirements. While an HL7 standard exists for specifying how the links should be constructed, there is no guidance on what links to construct. Collectively, the authors manage four infobutton systems that serve 16 institutions. The purpose of this paper is to publish our experience with linking various resources and specifying particular criteria that can be used by infobutton managers to select resources that are most relevant for a given situation. This experience can be used directly by those wishing to customize their own EHRs, for example by using the OpenInfobutton infobutton manager and its configuration tool, the Librarian Infobutton Tailoring Environment.

A study of diverse clinical decision support rule authoring environments and requirements for integration.

BACKGROUND: Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules. METHODS: The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools. RESULTS: While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users. CONCLUSIONS: A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Adverse drug events caused by serious medication administration errors.

OBJECTIVE: To determine how often serious or life-threatening medication administration errors with the potential to cause harm (potential adverse drug events) result in actual harm (adverse drug events (ADEs)) in the hospital setting. DESIGN: Retrospective chart review of clinical events following observed medication administration errors. BACKGROUND: Medication errors are common at the medication administration stage for inpatients. While many errors can cause harm, it is unclear exactly how often. METHODS: In a previous study where 14 041 medication administrations were directly observed, 1271 medication administration errors were discovered, of which 133 had the potential to cause serious or life-threatening harm and were considered serious or life-threatening potential adverse drug events. As a follow-up, clinical reviewers conducted detailed chart review of serious or life-threatening potential ADEs to determine if they caused an ADE. Reviewers assessed severity of the ADE and attribution to the error. RESULTS: Ten (7.5% (95% CI 6.98 to 8.01)) actual ADEs resulted from the 133 serious and life-threatening potential ADEs, of which 6 resulted in significant, three in serious, and one life threatening injury. Therefore 4 (3% (95% CI 2.12 to 3.6)) of serious or life threatening potential ADEs led to serious or life threatening ADEs. Half of the ADEs were caused by dosage or monitoring errors for anti-hypertensives. CONCLUSIONS: Unintercepted potential ADEs at the medication administration stage can cause serious patient harm. At hospitals where 6 million doses are administered per year, about 4000 preventable ADEs would be attributable to medication administration errors annually.

Fatty emaciation: a case report of suspected fat overload syndrome in oral refeeding.

Refeeding syndrome has been observed in patients receiving nutrition after a prolonged period of malnourishment and is characterized by multiple metabolic derangements. Besides hypophosphatemia and hypoglycemia, lipemia has been described in association with parenteral nutrition administration to the malnourished. The authors describe one anorexic patient who developed lipemia during oral refeeding, followed by a precipitous drop in hematocrit suggestive of fat overload syndrome.

Implementations of the HL7 Context-Aware Knowledge Retrieval ("Infobutton") Standard: challenges, strengths, limitations, and uptake.

UNLABELLED: To support clinical decision-making, computerized information retrieval tools known as "infobuttons" deliver contextually-relevant knowledge resources into clinical information systems. The Health Level Seven International (HL7) Context-Aware Knowledge Retrieval (Infobutton) Standard specifies a standard mechanism to enable infobuttons on a large scale. OBJECTIVE: To examine the experience of organizations in the course of implementing the HL7 Infobutton Standard. METHOD: Cross-sectional online survey and in-depth phone interviews. RESULTS: A total of 17 organizations participated in the study. Analysis of the in-depth interviews revealed 20 recurrent themes. Implementers underscored the benefits, simplicity, and flexibility of the HL7 Infobutton Standard. Yet, participants voiced the need for easier access to standard specifications and improved guidance to beginners. Implementers predicted that the Infobutton Standard will be widely or at least fairly well adopted in the next 5 years, but uptake will depend largely on adoption among electronic health record (EHR) vendors. To accelerate EHR adoption of the Infobutton Standard, implementers recommended HL7-compliant infobutton capabilities to be included in the United States Meaningful Use Certification Criteria for EHR systems. LIMITATIONS: Opinions and predictions should be interpreted with caution, since all the participant organizations have successfully implemented the standard and over half of the organizations were actively engaged in the development of the standard. CONCLUSION: Overall, implementers reported a very positive experience with the HL7 Infobutton Standard. Despite indications of increasing uptake, measures should be taken to stimulate adoption of the Infobutton Standard among EHR vendors. Widespread adoption of the Infobutton Standard has the potential to bring contextually relevant clinical decision support content into the healthcare provider workflow.

Supratherapeutic dosing of acetaminophen among hospitalized patients.

BACKGROUND: We investigated acetaminophen use and identify factors contributing to supratherapeutic dosing of acetaminophen in hospitalized patients. METHODS: We retrospectively reviewed the electronic health records of adult patients who were admitted to 2 academic tertiary care hospitals (hospital A amd hospital B) from June 1, 2010, to August 31, 2010, and who received acetaminophen during their hospitalization. Patients' acetaminophen administration records (including drug name, dose, administration time, hospital units, etc), demographic data, diagnoses, and results from liver function tests were obtained. The main outcome measures included acetaminophen exposure rate and supratherapeutic dosing rate among hospitalized patients, hazard ratios (HRs) and 95% confidence intervals (CIs) for risk factors for supratherapeutic dosing, and changes in liver function test before and after supratherapeutic dosing. RESULTS: A total of 14 411 patients (60.7%) were exposed to acetaminophen, of whom 955 (6.6%) exceeded the 4 g per day maximum recommended dose. In addition, 22.3% of patients who were 65 years or older and 17.6% of patients with chronic liver diseases exceeded the recommended limit of 3 g per day. Patients receiving excessive doses of acetaminophen tended to have significant alkaline phosphatase elevations, although causal relationship cannot be concluded. A significantly higher risk of supratherapeutic dosing was observed in hospital A (HR, 1.6 [95% CI, 1.4-1.8]), white patients (HR, 1.5 [95% CI, 1.3-1.7]), patients diagnosed as having osteoarthritis (HR, 1.4 [95% CI, 1.3-1.6]), and those who received scheduled administrations (HR, 16.6 [95% CI, 13.5-20.6]), multiple product formulations (HR, 2.4 [95% CI 2.0-2.9]), or the 500-mg strength formulation (HR, 1.9 [95% CI, 1.5-2.3]). A lower risk was found for pro re nata (as needed) administrations (HR, 0.7 [95% CI, 0.6-0.9]) and in nonsurgical and non­intensive care units (HR, 0.6 [95% CI, 0.5-0.7]). CONCLUSIONS: Supratherapeutic dosing of acetaminophen was significantly associated with multiple factors. Interventions to reduce the incidence of some risk factors may prevent supratherapeutic acetaminophen dosing in hospitalized patients.

A legal framework to enable sharing of Clinical Decision Support knowledge and services across institutional boundaries.

The goal of the CDS Consortium (CDSC) is to assess, define, demonstrate, and evaluate best practices for knowledge management and clinical decision support in healthcare information technology at scale - across multiple ambulatory care settings and Electronic Health Record technology platforms. In the course of the CDSC research effort, it became evident that a sound legal foundation was required for knowledge sharing and clinical decision support services in order to address data sharing, intellectual property, accountability, and liability concerns. This paper outlines the framework utilized for developing agreements in support of sharing, accessing, and publishing content via the CDSC Knowledge Management Portal as well as an agreement in support of deployment and consumption of CDSC developed web services in the context of a research project under IRB oversight.

A multi-layered framework for disseminating knowledge for computer-based decision support.

BACKGROUND: There are several challenges in encoding guideline knowledge in a form that is portable to different clinical sites, including the heterogeneity of clinical decision support (CDS) tools, of patient data representations, and of workflows. METHODS: We have developed a multi-layered knowledge representation framework for structuring guideline recommendations for implementation in a variety of CDS contexts. In this framework, guideline recommendations are increasingly structured through four layers, successively transforming a narrative text recommendation into input for a CDS system. We have used this framework to implement rules for a CDS service based on three guidelines. We also conducted a preliminary evaluation, where we asked CDS experts at four institutions to rate the implementability of six recommendations from the three guidelines. CONCLUSION: The experience in using the framework and the preliminary evaluation indicate that this approach has promise in creating structured knowledge, to implement in CDS systems, that is usable across organizations.

Governance for clinical decision support: case studies and recommended practices from leading institutions.

OBJECTIVE: Clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety; however, effective implementation of CDS requires effective clinical and technical governance structures. The authors sought to determine the range and variety of these governance structures and identify a set of recommended practices through observational study. DESIGN: Three site visits were conducted at institutions across the USA to learn about CDS capabilities and processes from clinical, technical, and organizational perspectives. Based on the results of these visits, written questionnaires were sent to the three institutions visited and two additional sites. Together, these five organizations encompass a variety of academic and community hospitals as well as small and large ambulatory practices. These organizations use both commercially available and internally developed clinical information systems. MEASUREMENTS: Characteristics of clinical information systems and CDS systems used at each site as well as governance structures and content management approaches were identified through extensive field interviews and follow-up surveys. RESULTS: Six recommended practices were identified in the area of governance, and four were identified in the area of content management. Key similarities and differences between the organizations studied were also highlighted. CONCLUSION: Each of the five sites studied contributed to the recommended practices presented in this paper for CDS governance. Since these strategies appear to be useful at a diverse range of institutions, they should be considered by any future implementers of decision support.

Creating shareable decision support services: an interdisciplinary challenge.

Creating shareable decision support services is a complex task requiring effort from multiple interdisciplinary role players with a wide variety of experience and expertise. The CDS Consortium research project has developed such a service, defining a multi-layer representation of knowledge and building upon an architectural service design created at Partners Health Care, and is demonstrating its use in both a local and an external institutional setting. The process was iterative, and we encountered unexpected requirements based on decisions made at various points. We report in this paper on challenges we faced while pursuing this research: knowledge representation and modeling, data interchange and standards adoption, the process of getting agreement on content, logistics of integrating into a system that already has multiple CDS interventions, legal issues around privacy and access, inter-team communication and organization.

The morningside initiative: collaborative development of a knowledge repository to accelerate adoption of clinical decision support.

The Morningside Initiative is a public-private activity that has evolved from an August, 2007, meeting at the Morningside Inn, in Frederick, MD, sponsored by the Telemedicine and Advanced Technology Research Center (TATRC) of the US Army Medical Research Materiel Command. Participants were subject matter experts in clinical decision support (CDS) and included representatives from the Department of Defense, Veterans Health Administration, Kaiser Permanente, Partners Healthcare System, Henry Ford Health System, Arizona State University, and the American Medical Informatics Association (AMIA). The Morningside Initiative was convened in response to the AMIA Roadmap for National Action on Clinical Decision Support and on the basis of other considerations and experiences of the participants. Its formation was the unanimous recommendation of participants at the 2007 meeting which called for creating a shared repository of executable knowledge for diverse health care organizations and practices, as well as health care system vendors. The rationale is based on the recognition that sharing of clinical knowledge needed for CDS across organizations is currently virtually non-existent, and that, given the considerable investment needed for creating, maintaining and updating authoritative knowledge, which only larger organizations have been able to undertake, this is an impediment to widespread adoption and use of CDS. The Morningside Initiative intends to develop and refine (1) an organizational framework, (2) a technical approach, and (3) CDS content acquisition and management processes for sharing CDS knowledge content, tools, and experience that will scale with growing numbers of participants and can be expanded in scope of content and capabilities. Intermountain Healthcare joined the initial set of participants shortly after its formation. The efforts of the Morningside Initiative are intended to serve as the basis for a series of next steps in a national agenda for CDS. It is based on the belief that sharing of knowledge can be highly effective as is the case in other competitive domains such as genomics. Participants in the Morningside Initiative believe that a coordinated effort between the private and public sectors is needed to accomplish this goal and that a small number of highly visible and respected health care organizations in the public and private sector can lead by example. Ultimately, a future collaborative knowledge sharing organization must have a sustainable long-term business model for financial support.

A large-scale knowledge management method based on the analysis of the use of online knowledge resources.

UNLABELLED: Online health knowledge resources can be integrated into electronic health record systems using decision support tools known as "infobuttons." In this study we describe a knowledge management method based on the analysis of knowledge resource use via infobuttons in multiple institutions. METHODS: We conducted a two-phase analysis of laboratory test infobutton sessions at three healthcare institutions accessing two knowledge resources. The primary study measure was session coverage, i.e. the rate of infobutton sessions in which resources retrieved relevant content. RESULTS: In Phase One, resources covered 78.5% of the study sessions. In addition, a subset of 38 noncovered tests that most frequently raised questions was identified. In Phase Two, content development guided by the outcomes of Phase One resulted in a 4% average coverage increase. CONCLUSION: The described method is a valuable approach to large-scale knowledge management in rapidly changing domains.