Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics.

OBJECTIVE: The Leadership in Analytics and Data Science (LEADS) course was evaluated for effectiveness. LEADS was a 6-month program for working biomedical and health informatics (BMHI) professionals designed to improve analytics skills, knowledge of enterprise applications, data stewardship, and to foster an analytics community of practice through lectures, hands-on skill building workshops, networking events, and small group projects. METHODS: The effectiveness of the LEADS course was evaluated using the Kirkpatrick Model by assessing pre- and postcourse knowledge, analytics capabilities, goals, practice, class lecture reaction, and change in the size of participant professional networks. Differences in pre- and postcourse responses were analyzed with a Wilcoxon signed rank test to determine significance, and effect sizes were computed using a z-statistic. RESULTS: Twenty-nine students completed the course with 96% of respondents reporting that they were "very" or "extremely" likely to recommend the course. Participants reported improvement in several analytics capabilities including Epic data warehousing (p = 0.017), institutional review board policy (p = 0.005), and data stewardship (p = 0.007). Changes in practice patterns mirrored those in self-reported capability. On average, the participant professional network doubled. CONCLUSION: LEADS was the first course targeted to working BMHI professional at a large academic medical center to have a formal effectiveness evaluation be published in the literature. The course achieved the goals of expansion of BMHI knowledge, skills, and professional networks. The LEADS course provides a template for continuing education of working BMHI professionals.

Patient-Centered Radiology with FHIR: an Introduction to the Use of FHIR to Offer Radiology a Clinically Integrated Platform.

Fast Healthcare Interoperability Resources (FHIR) is an open interoperability standard that allows external software to quickly search for and access clinical information from the electronic medical record (EMR) in a method that is developer-friendly, using current internet technology standards. In this article, we highlight the new FHIR standard and illustrate how FHIR can be used to offer the field of radiology a more clinically integrated and patient-centered system, opening the EMR to external radiology software in ways unfeasible with traditional standards. We explain how to construct FHIR queries relevant to medical imaging using the Society for Imaging Informatics in Medicine (SIIM) Hackathon application programming interface (API), provide sample queries for use, and suggest solutions to offer a patient-centered, rather than an image-centered, workflow that remains clinically relevant.

The Impact of Imaging Informatics Fellowships.

Imaging informatics (II) is an area within clinical informatics that is particularly important in the field of radiology. Provider groups have begun employing dedicated radiologist-informaticists to bridge medical, information technology and administrative functions, and academic institutions are meeting this demand through formal II fellowships. However, little is known about how these programs influence graduates' careers and perceptions about professional development. We electronically surveyed 26 graduates from US II fellowships and consensus leaders in the II community-many of whom were subspecialty diagnostic radiologists (68%) employed within academic institutions (48%)-about the perceived impact of II fellowships on career development and advancement. All graduates felt that II fellowship made them more valuable to employers, with the majority of reporting ongoing II roles (78%) and continued used of competencies (61%) and skills (56%) gained during fellowship in their current jobs. Other key benefits included access to mentors, protected time for academic work, networking opportunities, and positive impacts of annual compensation. Of respondents without II fellowship training, all would recommend fellowships to current trainees given the ability to gain a "still rare" but "essential skill set" that is "critical for future leaders in radiology" and "better job opportunities." While some respondents felt that II fellowships needed further formalization and standardization, most (85%) disagreed with requiring a 2-year II fellowship in order to qualify for board certification in clinical informatics. Instead, most believed that fellowships should be integrated with clinical residency or fellowship training while preserving formal didactics and unstructured project time. More work is needed to understand existing variations in II fellowship training structure and identify the optimal format for programs targeted at radiologists.

Changes to stage 1 meaningful use in 2014: impact on radiologists.

The goal of this work is to provide radiologists an update regarding changes to stage 1 of meaningful use in 2014. These changes were promulgated in the final rulemaking released by the Centers for Medicare and Medicaid Services and the Office of the National Coordinator for Health Information Technology in September 2012. Under the new rules, radiologists are exempt from meaningful use penalties provided that they are listed as radiologists under the Provider Enrollment, Chain and Ownership System (PECOS). A major caveat is that this exemption can be removed at any time. Additional concerns are discussed in the main text. Additional changes discussed include software editions independent of meaningful use stage (i.e., 2011 edition versus 2014 edition), changes to the definition of certified electronic health record technology (CEHRT), and changes to specific measures and exemptions to those measures. The new changes regarding stage 1 add complexity to an already complex program, but overall make achieving meaningful use a win-win situation for radiologists. There are no penalties for failure and incentive payments for success. The cost of upgrading to CEHRT may be much less than the incentive payments, adding a potential new source of revenue. Additional benefits may be realized if the radiology department can build upon a modern electronic health record to improve their practice and billing patterns. Meaningful use and electronic health records represent an important evolutionary step in US healthcare, and it is imperative that radiologists are active participants in the process.

Guide to effective quality improvement reporting in radiology.

Substantial societal investments in biomedical research are contributing to an explosion in knowledge that the health delivery system is struggling to effectively implement. Managing this complexity requires ingenuity, research and development, and dedicated resources. Many innovative solutions can be found in quality improvement (QI) activities, defined as the "systematic, data-guided activities designed to bring about immediate, positive changes in the delivery of healthcare in particular settings." QI shares many similarities with biomedical research, but also differs in several important ways. Inclusion of QI in the peer-reviewed literature is needed to foster its advancement through the dissemination, testing, and refinement of theories, methods, and applications. QI methods and reporting standards are less mature in health care than those of biomedical research. A lack of widespread understanding and consensus regarding the purpose of publishing QI-related material also exists. In this document, guidance is provided in evaluating quality of QI-related material and in determining priority of submitted material for publication.

Building blocks for a clinical imaging informatics environment.

Over the past 20 years, imaging informatics has been driven by the widespread adoption of radiology information and picture archiving and communication and speech recognition systems. These three clinical information systems are commonplace and are intuitive to most radiologists as they replicate familiar paper and film workflow. So what is next? There is a surge of innovation in imaging informatics around advanced workflow, search, electronic medical record aggregation, dashboarding, and analytics tools for quality measures (Nance et al., AJR Am J Roentgenol 200:1064-1070, 2013). The challenge lies in not having to rebuild the technological wheel for each of these new applications but instead attempt to share common components through open standards and modern development techniques. The next generation of applications will be built with moving parts that work together to satisfy advanced use cases without replicating databases and without requiring fragile, intense synchronization from clinical systems. The purpose of this paper is to identify building blocks that can position a practice to be able to quickly innovate when addressing clinical, educational, and research-related problems. This paper is the result of identifying common components in the construction of over two dozen clinical informatics projects developed at the University of Maryland Radiology Informatics Research Laboratory. The systems outlined are intended as a mere foundation rather than an exhaustive list of possible extensions.

Cognitive and system factors contributing to diagnostic errors in radiology.

OBJECTIVE: In this article, we describe some of the cognitive and system-based sources of detection and interpretation errors in diagnostic radiology and discuss potential approaches to help reduce misdiagnoses. CONCLUSION: Every radiologist worries about missing a diagnosis or giving a false-positive reading. The retrospective error rate among radiologic examinations is approximately 30%, with real-time errors in daily radiology practice averaging 3-5%. Nearly 75% of all medical malpractice claims against radiologists are related to diagnostic errors. As medical reimbursement trends downward, radiologists attempt to compensate by undertaking additional responsibilities to increase productivity. The increased workload, rising quality expectations, cognitive biases, and poor system factors all contribute to diagnostic errors in radiology. Diagnostic errors are underrecognized and underappreciated in radiology practice. This is due to the inability to obtain reliable national estimates of the impact, the difficulty in evaluating effectiveness of potential interventions, and the poor response to systemwide solutions. Most of our clinical work is executed through type 1 processes to minimize cost, anxiety, and delay; however, type 1 processes are also vulnerable to errors. Instead of trying to completely eliminate cognitive shortcuts that serve us well most of the time, becoming aware of common biases and using metacognitive strategies to mitigate the effects have the potential to create sustainable improvement in diagnostic errors.

Cloud computing in medical imaging.

Over the past century technology has played a decisive role in defining, driving, and reinventing procedures, devices, and pharmaceuticals in healthcare. Cloud computing has been introduced only recently but is already one of the major topics of discussion in research and clinical settings. The provision of extensive, easily accessible, and reconfigurable resources such as virtual systems, platforms, and applications with low service cost has caught the attention of many researchers and clinicians. Healthcare researchers are moving their efforts to the cloud, because they need adequate resources to process, store, exchange, and use large quantities of medical data. This Vision 20/20 paper addresses major questions related to the applicability of advanced cloud computing in medical imaging. The paper also considers security and ethical issues that accompany cloud computing.

The future of the radiology information system.

OBJECTIVE: Today in the hospital setting, several functions of the radiology information system (RIS), including order entry, patient registration, report repository, and the physician directory, have moved to enterprise electronic medical records. Some observers might conclude that the RIS is going away. In this article, we contend that because of the maturity of the RIS market compared with other areas of the health care enterprise, radiology has a unique opportunity to innovate. CONCLUSION: While most of the hospital enterprise spends the next several years going through the digital transformation converting from paper to a digital format, radiology can leap ahead in its use of analytics and information technology. This article presents a summary of new RIS functions still maturing and open to innovation in the RIS market.

Reporting of critical findings in neuroradiology.

OBJECTIVE: The objective of our study was to assess compliance among academic neuroradiologists in reporting institutionally derived critical findings. MATERIALS AND METHODS: We analyzed 3054 neuroradiology CT and MRI reports generated in 1 month. Reports were categorized by whether or not they contained a critical finding based on a previously established list. The reports were subcategorized by whether the reporting neuroradiologist flagged the report as containing a critical finding and whether the radiologist verbally communicated the critical finding to the referring clinician. Reports were divided into day or night categories and the frequency of critical findings for each time period was calculated. RESULTS: Of the 3054 reports included in this study, 301 (9.9%) had critical findings. Of those 301 reports, 233 (77.4%) were flagged and the referring clinician was called. Of the remaining 68 reports with critical findings, the reporting radiologist did not call the clinician about 35.3% of them (24/68). Of the 2753 reports without critical findings, 2658 (96.5%) were appropriately not flagged and the clinician was not called. However, radiologists called clinicians about 3.5% (95/2753) of the reports without critical findings and erroneously flagged 68.4% (65/95) of those reports as critical. A majority of the cases with critical findings were reported at night (55.1%) despite the fact that 67.2% of the studies occurred during the day. CONCLUSION: Compliance with reporting and communicating critical findings must be monitored. Calling clinicians to report noncritical findings may result in unnecessary interruptions in work flow for radiologists and referring health care providers.

Medical imaging displays and their use in image interpretation.

The adequate and repeatable performance of the image display system is a key element of information technology platforms in a modern radiology department. However, despite the wide availability of high-end computing platforms and advanced color and gray-scale monitors, the quality and properties of the final displayed medical image may often be inadequate for diagnostic purposes if the displays are not configured and maintained properly. In this article-an expanded version of the Radiological Society of North America educational module "Image Display"-the authors discuss fundamentals of image display hardware, quality control and quality assurance processes for optimal image interpretation settings, and parameters of the viewing environment that influence reader performance. Radiologists, medical physicists, and other allied professionals should strive to understand the role of display technology and proper usage for a quality radiology practice. The display settings and display quality control and quality assurance processes described in this article can help ensure high standards of perceived image quality and image interpretation accuracy.

Unbiased review of digital diagnostic images in practice: informatics prototype and pilot study.

RATIONALE AND OBJECTIVES: Clinical and contextual information associated with images may influence how radiologists draw diagnostic inferences, highlighting the need to control multiple sources of bias in the methodologic design of investigations involving radiologic interpretation. In the past, manual control methods to mask review films presented in practice have been used to reduce potential interpretive bias associated with differences between viewing images for patient care and reviewing images for the purposes of research, education, and quality improvement. These manual precedents from the film era raise the question whether similar methods to reduce bias can be implemented in the modern digital environment. MATERIALS AND METHODS: A prototype application, CreateAPatient, was built for masking review case presentations within one institution's production radiology information system and picture archiving and communication system. To test whether CreateAPatient could be used to mask review images presented in practice, six board-certified radiologists participated in a pilot study. During pilot testing, seven digital chest radiographs, known to contain lung nodules and associated with fictitious patient identifiers, were mixed into the routine workloads of the participating radiologists while they covered general evening call shifts. The aim was to test whether it was possible to mask the presentation of these review cases, both by probing the interpreting radiologists to report detection and by conducting a forced-choice experiment on a separate cohort of 20 radiologists and information technology professionals. RESULTS: None of the participating radiologists reported awareness of review activity, and forced-choice detection was less than predicted at chance, suggesting that radiologists were effectively blinded. In addition, no evidence was identified of review reports unsafely propagating beyond their intended scope or otherwise interfering with patient care, despite integration of these records within production electronic work flow systems. CONCLUSIONS: Information technology can facilitate the design of unbiased methods involving professional review of digital diagnostic images.

Certification of Imaging Informatics Professionals (CIIP): 2010 survey of diplomates.

The Certification for Imaging Informatics Professionals (CIIP) program is sponsored by the Society of Imaging Informatics in Medicine and the American Registry of Radiologic Technologists through the American Board of Imaging Informatics. In 2005, a survey was conducted of radiologists, technologists, information technology specialists, corporate information officers, and radiology administrators to identify the competencies and skill set that would define a successful PACS administrator. The CIIP examination was created in 2007 in response to the need for an objective way to test for such competencies, and there have been 767 professionals who have been certified through this program to date. The validity of the psychometric integrity of the examination has been previously established. In order to further understand the impact and future direction of the CIIP certification on diplomats, a survey was conducted in 2010. This paper will discuss the results of the survey.

Five levels of PACS modularity: integrating 3D and other advanced visualization tools.

The current array of PACS products and 3D visualization tools presents a wide range of options for applying advanced visualization methods in clinical radiology. The emergence of server-based rendering techniques creates new opportunities for raising the level of clinical image review. However, best-of-breed implementations of core PACS technology, volumetric image navigation, and application-specific 3D packages will, in general, be supplied by different vendors. Integration issues should be carefully considered before deploying such systems. This work presents a classification scheme describing five tiers of PACS modularity and integration with advanced visualization tools, with the goals of characterizing current options for such integration, providing an approach for evaluating such systems, and discussing possible future architectures. These five levels of increasing PACS modularity begin with what was until recently the dominant model for integrating advanced visualization into the clinical radiologist's workflow, consisting of a dedicated stand-alone post-processing workstation in the reading room. Introduction of context-sharing, thin clients using server-based rendering, archive integration, and user-level application hosting at successive levels of the hierarchy lead to a modularized imaging architecture, which promotes user interface integration, resource efficiency, system performance, supportability, and flexibility. These technical factors and system metrics are discussed in the context of the proposed five-level classification scheme.

Developing and verifying the psychometric integrity of the certification examination for imaging informatics professionals.

The American Board of Imaging Informatics (ABII) was founded in 2005 by the Society of Imaging Informatics in Medicine (SIIM) and the American Registry of Radiologic Technologists (ARRT). ABII's mission is to enhance patient care, professionalism, and competence in imaging informatics. This is accomplished primarily through the development and administration of a certification examination. The creation of the exam has been an exercise in open community involvement with SIIM providing access to the PACS community and ARRT providing skilled psychometric support to ensure a balanced and comprehensive examination. The process to generate the exam required several years and the efforts of dozens of subject matter experts active who volunteered to submit and validate questions for the examination. This article describes the organizational and statistical processes used to generate test items, assemble test forms, set performance standards, and validate test scores.

Informatics in radiology: automated Web-based graphical dashboard for radiology operational business intelligence.

Radiology departments today are faced with many challenges to improve operational efficiency, performance, and quality. Many organizations rely on antiquated, paper-based methods to review their historical performance and understand their operations. With increased workloads, geographically dispersed image acquisition and reading sites, and rapidly changing technologies, this approach is increasingly untenable. A Web-based dashboard was constructed to automate the extraction, processing, and display of indicators and thereby provide useful and current data for twice-monthly departmental operational meetings. The feasibility of extracting specific metrics from clinical information systems was evaluated as part of a longer-term effort to build a radiology business intelligence architecture. Operational data were extracted from clinical information systems and stored in a centralized data warehouse. Higher-level analytics were performed on the centralized data, a process that generated indicators in a dynamic Web-based graphical environment that proved valuable in discussion and root cause analysis. Results aggregated over a 24-month period since implementation suggest that this operational business intelligence reporting system has provided significant data for driving more effective management decisions to improve productivity, performance, and quality of service in the department.

Quality control management and communication between radiologists and technologists.

PURPOSE: The greatest barrier to quality control (QC) in the digital imaging environment is the lack of communication and documentation between those who interpret images and those who acquire them. Paper-based QC methods are insufficient in a digital image management system. Problem work flow must be incorporated into reengineering efforts when migrating to a digital practice. The authors implemented a Web-based QC feedback tool to document and facilitate the communication of issues identified by radiologists. The goal was to promote a responsive and constructive tool that contributes to a culture of quality. The hypothesis was that by making it easier for radiologists to submit quality issues, the number of QC issues submitted would increase. METHODS: The authors integrated their Web-based quality tracking system with a clinical picture archiving and communication system so that radiologists could report quality issues without disrupting clinical work flow. Graphical dashboarding techniques aid supervisors in using this database to identify the root causes of different types of issues. RESULTS: Over the initial 12-month rollout period, starting in the general section, the authors recorded 20 times more QC issues submitted by radiologists, accompanied by a rise in technologists' responsiveness to QC issues. For technologists with high numbers of QC issues, the incorporation of data from this tracking system proved useful in performance appraisals and in driving individual improvement. CONCLUSION: This tool is an example of the types of information technology innovations that can be leveraged to support QC in the digital imaging environment. Initial data suggest that the result is not only an improvement in quality but higher levels of satisfaction for both radiologists and technologists.

Novel, Web-based, information-exploration approach for improving operating room logistics and system processes.

Routine clinical information systems now have the ability to gather large amounts of data that surgical managers can access to create a seamless and proactive approach to streamlining operations and minimizing delays. The challenge lies in aggregating and displaying these data in an easily accessible format that provides useful, timely information on current operations. A Web-based, graphical dashboard is described in this study, which can be used to interpret clinical operational data, allow managers to see trends in data, and help identify inefficiencies that were not apparent with more traditional, paper-based approaches. The dashboard provides a visual decision support tool that assists managers in pinpointing areas for continuous quality improvement. The limitations of paper-based techniques, the development of the automated display system, and key performance indicators in analyzing aggregate delays, time, specialties, and teamwork are reviewed. Strengths, weaknesses, opportunities, and threats associated with implementing such a program in the perioperative environment are summarized.