Implementation barriers and considerations for recommending and administering the human papillomavirus (HPV) vaccination in oncology settings.

BACKGROUND: Survivors of childhood and adolescent cancer experience low human papillomavirus (HPV) vaccination rates-a crucial form of cancer prevention. Oncology provider recommendations may increase young survivors HPV vaccine intent, but HPV vaccination is not typically provided in the oncology setting. Thus, we explored the implementation barriers of providing the HPV vaccine in oncology. METHODS: We interviewed oncology providers in a variety of specialty areas about their perceptions of the HPV vaccine and to explore barriers to recommending and administering the vaccine in their clinics. Interviews were audio recorded, quality checked, and thematically analyzed. Emergent themes were then mapped onto the Capability, Opportunity, Motivation, and Behavior (COM-B) Model and the Theoretical Domains Framework. RESULTS: A total of N=24 oncology providers were interviewed. Most provided direct clinical care (87.5%) and most commonly specialized in pediatric oncology (20.8%), medical oncology (16.7%), bone marrow transplant (16.7%), and nurse coordination (16.7%). Two themes emerged within each COM-B domain. Capability: 1) educational barriers to HPV vaccination and 2) complicated post treatment HPV vaccination guidelines. MOTIVATION: 1) perceived importance of HPV vaccine and 2) concern about blurred scope of practice. OPPORTUNITY:  1) hospital administration and time concern barriers and 2) clinical workflow integration concerns. CONCLUSION: Implementing HPV vaccination in the oncology setting has the potential to increase HPV vaccination rates among young survivors. Multi-level barriers to providing the HPV vaccine in the oncology setting were identified by participants. Leveraging existing implementation strategies may be an effective way to mitigate provider identified barriers and increase vaccination rates.

Evaluation in Life Cycle of Information Technology (ELICIT) framework: Supporting the innovation life cycle from business case assessment to summative evaluation.

OBJECTIVE: Our objective was to develop an evaluation framework for electronic health record (EHR)-integrated innovations to support evaluation activities at each of four information technology (IT) life cycle phases: planning, development, implementation, and operation. METHODS: The evaluation framework was developed based on a review of existing evaluation frameworks from health informatics and other domains (human factors engineering, software engineering, and social sciences); expert consensus; and real-world testing in multiple EHR-integrated innovation studies. RESULTS: The resulting Evaluation in Life Cycle of IT (ELICIT) framework covers four IT life cycle phases and three measure levels (society, user, and IT). The ELICIT framework recommends 12 evaluation steps: (1) business case assessment; (2) stakeholder requirements gathering; (3) technical requirements gathering; (4) technical acceptability assessment; (5) user acceptability assessment; (6) social acceptability assessment; (7) social implementation assessment; (8) initial user satisfaction assessment; (9) technical implementation assessment; (10) technical portability assessment; (11) long-term user satisfaction assessment; and (12) social outcomes assessment. DISCUSSION: Effective evaluation requires a shared understanding and collaboration across disciplines throughout the entire IT life cycle. In contrast with previous evaluation frameworks, the ELICIT framework focuses on all phases of the IT life cycle across the society, user, and IT levels. Institutions seeking to establish evaluation programs for EHR-integrated innovations could use our framework to create such shared understanding and justify the need to invest in evaluation. CONCLUSION: As health care undergoes a digital transformation, it will be critical for EHR-integrated innovations to be systematically evaluated. The ELICIT framework can facilitate these evaluations.

Implementing lung cancer screening in primary care: needs assessment and implementation strategy design.

Lung cancer screening with low-dose computed tomography (CT) could help avert thousands of deaths each year. Since the implementation of screening is complex and underspecified, there is a need for systematic and theory-based strategies. Explore the implementation of lung cancer screening in primary care, in the context of integrating a decision aid into the electronic health record. Design implementation strategies that target hypothesized mechanisms of change and context-specific barriers. The study had two phases. The Qualitative Analysis phase included semi-structured interviews with primary care physicians to elicit key task behaviors (e.g., ordering a low-dose CT) and understand the underlying behavioral determinants (e.g., social influence). The Implementation Strategy Design phase consisted of defining implementation strategies and hypothesizing causal pathways to improve screening with a decision aid. Three key task behaviors and four behavioral determinants emerged from 14 interviews. Implementation strategies were designed to target multiple levels of influence. Strategies included increasing provider self-efficacy toward performing shared decision making and using the decision aid, improving provider performance expectancy toward ordering a low-dose CT, increasing social influence toward performing shared decision making and using the decision aid, and addressing key facilitators to using the decision aid. This study contributes knowledge about theoretical determinants of key task behaviors associated with lung cancer screening. We designed implementation strategies according to causal pathways that can be replicated and tested at other institutions. Future research is needed to evaluate the effectiveness of these strategies and to determine the contexts in which they can be effectively applied.

Workflow analysis for design of an electronic health record-based tobacco cessation intervention in community health centers.

OBJECTIVE: Tobacco use is the leading cause of preventable morbidity and mortality in the United States. Quitlines are effective telephone-based tobacco cessation services but are underutilized. The goal of this project was to describe current clinical workflows for Quitline referral and design an optimal electronic health record (EHR)-based workflow for Ask-Advice-Connect (AAC), an evidence-based intervention to increase Quitline referrals. MATERIALS AND METHODS: Ten Community Health Center systems (CHC), which use three different EHRs, participated in this study. Methods included: 9 group discussions with CHC leaders; 33 observations/interviews of clinical teams' workflow; surveys with 57 clinical staff; and assessment of the EHR ecosystem in each CHC. Data across these methods were integrated and coded according to the Fit between Individual, Task, Technology and Environment (FITTE) framework. The current and optimal workflow were notated using Business Process Modelling Notation. We compared the requirements of the optimal workflow with EHR capabilities. RESULTS: Current workflows are inefficient in data collection, variable in who, how, and when tobacco cessation advice and referral are enacted, and lack communication between referring clinics and the Quitline. In the optimal workflow, medical assistants deliver a standardized AAC intervention during the visit intake. Referrals are submitted electronically, and there is bidirectional communication between the clinic and Quitline. We implemented AAC within all three EHRs; however, deviations from the optimal workflow were necessary. CONCLUSION: Current workflows for Quitline referral are inefficient and ineffective. We propose an optimal workflow and discuss improvements in EHR capabilities that would improve the implementation of AAC.

Establishing a multidisciplinary initiative for interoperable electronic health record innovations at an academic medical center.

OBJECTIVE: To establish an enterprise initiative for improving health and health care through interoperable electronic health record (EHR) innovations. MATERIALS AND METHODS: We developed a unifying mission and vision, established multidisciplinary governance, and formulated a strategic plan. Key elements of our strategy include establishing a world-class team; creating shared infrastructure to support individual innovations; developing and implementing innovations with high anticipated impact and a clear path to adoption; incorporating best practices such as the use of Fast Healthcare Interoperability Resources (FHIR) and related interoperability standards; and maximizing synergies across research and operations and with partner organizations. RESULTS: University of Utah Health launched the ReImagine EHR initiative in 2016. Supportive infrastructure developed by the initiative include various FHIR-related tooling and a systematic evaluation framework. More than 10 EHR-integrated digital innovations have been implemented to support preventive care, shared decision-making, chronic disease management, and acute clinical care. Initial evaluations of these innovations have demonstrated positive impact on user satisfaction, provider efficiency, and compliance with evidence-based guidelines. Return on investment has included improvements in care; over $35 million in external grant funding; commercial opportunities; and increased ability to adapt to a changing healthcare landscape. DISCUSSION: Key lessons learned include the value of investing in digital innovation initiatives leveraging FHIR; the importance of supportive infrastructure for accelerating innovation; and the critical role of user-centered design, implementation science, and evaluation. CONCLUSION: EHR-integrated digital innovation initiatives can be key assets for enhancing the EHR user experience, improving patient care, and reducing provider burnout.

Leveraging Artificial Intelligence to Improve Chronic Disease Care: Methods and Application to Pharmacotherapy Decision Support for Type-2 Diabetes Mellitus.

OBJECTIVES: Artificial intelligence (AI), including predictive analytics, has great potential to improve the care of common chronic conditions with high morbidity and mortality. However, there are still many challenges to achieving this vision. The goal of this project was to develop and apply methods for enhancing chronic disease care using AI. METHODS: Using a dataset of 27,904 patients with diabetes, an analytical method was developed and validated for generating a treatment pathway graph which consists of models that predict the likelihood of alternate treatment strategies achieving care goals. An AI-driven clinical decision support system (CDSS) integrated with the electronic health record (EHR) was developed by encapsulating the prediction models in an OpenCDS Web service module and delivering the model outputs through a SMART on FHIR (Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources) web-based dashboard. This CDSS enables clinicians and patients to review relevant patient parameters, select treatment goals, and review alternate treatment strategies based on prediction results. RESULTS: The proposed analytical method outperformed previous machine-learning algorithms on prediction accuracy. The CDSS was successfully integrated with the Epic EHR at the University of Utah. CONCLUSION: A predictive analytics-based CDSS was developed and successfully integrated with the EHR through standards-based interoperability frameworks. The approach used could potentially be applied to many other chronic conditions to bring AI-driven CDSS to the point of care.

Feasibility and acceptability of a game-based symptom-reporting app for children with cancer: perspectives of children and parents.

BACKGROUND: Children with cancer have difficulty identifying and describing the multiple symptoms they experience during hospitalization and between clinical encounters. Mobile health resources, including apps, are potential solutions to support child-centric symptom reporting. This study evaluated the feasibility and acceptability of a newly developed game-based symptom-reporting app for school-age children with cancer. PROCEDURE: Nineteen school-age children (6-12 years of age) receiving treatment for cancer at a COG institution in the Intermountain West of the United States used a game-based symptom-reporting app between clinical visits. Feasibility was evaluated through a summary of actual days of app use and interaction with each of the app's features. Children and their parents participated in interviews regarding the app's acceptability. RESULTS: Children used the app a median of 4 days (range 1-12) and interacted most frequently with the symptom reporting and the drawing features. Children enjoyed aspects of the app that supported their creativity and provided choices. Parents endorsed the interactive nature of the app and the value of the child providing his/her own report. Both children and parents identified additional opportunities to enhance the child's user experience. CONCLUSION: Study results support the preliminary feasibility and acceptability of the app. Children's and parents' responses supported the developmental relevance of the app and its role in enhancing the child's autonomy and serving as an outlet for creativity. Future directions include optimizing the child user's experience and investigating the app's role as a resource to enhance shared decision-making for symptom management.

Patient-centered care and the electronic health record: exploring functionality and gaps.

OBJECTIVE: Healthcare systems have adopted electronic health records (EHRs) to support clinical care. Providing patient-centered care (PCC) is a goal of many healthcare systems. In this study, we sought to explore how existing EHR systems support PCC; defined as understanding the patient as a whole person, building relational connections between the clinician and patient, and supporting patients in health self-management. MATERIALS AND METHODS: We assessed availability of EHR functions consistent with providing PCC including patient goals and preferences, integrated care plans, and contextual and patient-generated data. We surveyed and then interviewed technical representatives and expert clinical users of 6 leading EHR systems. Questions focused on the availability of specific data and functions related to PCC (for technical representatives) and the clinical usefulness of PCC functions (for clinicians) in their EHR. RESULTS: Technical representatives (n = 6) reported that patient communication preferences, personalized indications for medications, and end of life preferences were functions implemented across 6 systems. Clinician users (n = 10) reported moderate usefulness of PCC functions (medians of 2-4 on a 5-pointy -35t scale), suggesting the potential for improvement across systems. Interviews revealed that clinicians do not have a shared conception of PCC. In many cases, data needed to deliver PCC was available in the EHR only in unstructured form. Data systems and functionality to support PCC are under development in these EHRs. DISCUSSION AND CONCLUSION: There are current gaps in PCC functionality in EHRs and opportunities to support the practice of PCC through EHR redesign.

Primary care perspectives on implementation of clinical trial recruitment.

INTRODUCTION: Poor clinical trial (CT) recruitment is a significant barrier to translating basic science discoveries into medical practice. Improving support for primary care provider (PCP) referral of patients to CTs may be an important part of the solution. However, implementing CT referral support in primary care is not only technically challenging, but also presents challenges at the person and organization levels. METHODS: The objectives of this study were (1) to characterize provider and clinical supervisor attitudes and perceptions regarding CT research, recruitment, and referrals in primary care and (2) to identify perceived workflow strategies and facilitators relevant to designing a technology-supported primary care CT referral program. Focus groups were conducted with PCPs, directors, and supervisors. RESULTS: Analysis indicated widespread support for the intrinsic scientific value of CTs, while at the same time deep concerns regarding protecting patient well-being, perceived loss of control when patients participate in trials, concern about the impact of point-of-care referrals on clinic workflow, the need for standard processes, and the need for CT information that enables referring providers to quickly confirm that the burdens are justified by the benefits at both patient and provider levels. PCP suggestions pertinent to implementing a CT referral decision support system are reported. CONCLUSION: The results from this work contribute to developing an implementation approach to support increased referral of patients to CTs.

QuitSMART Utah: an implementation study protocol for a cluster-randomized, multi-level Sequential Multiple Assignment Randomized Trial to increase Reach and Impact of tobacco cessation treatment in Community Health Centers.

BACKGROUND: Tobacco use remains the leading cause of death and disability in the USA and is disproportionately concentrated among low socioeconomic status (SES) populations. Community Health Centers (CHCs) are a key venue for reaching low SES populations with evidence-based tobacco cessation treatment such as Quitlines. Electronic health record (EHR)-based interventions at the point-of-care, text messaging (TM), and phone counseling have the potential to increase Quitline reach and are feasible to implement within CHCs. However, there is a lack of data to inform how, when, and in what combination these strategies should be implemented. The aims of this cluster-randomized trial are to evaluate multi-level implementation strategies to increase the Reach (i.e., proportion of tobacco-using patients who enroll in the Quitline) and Impact (i.e., Reach × Efficacy [efficacy is defined as the proportion of tobacco-using patients who enroll in Quitline treatment that successfully quit]) and to evaluate characteristics of healthcare system, providers, and patients that may influence tobacco-use outcomes. METHODS: This study is a multilevel, three-phase, Sequential Multiple Assignment Randomized Trial (SMART), conducted in CHCs (N = 33 clinics; N = 6000 patients). In the first phase, clinics will be randomized to two different EHR conditions. The second and third phases are patient-level randomizations based on prior treatment response. Patients who enroll in the Quitline receive no further interventions. In phase two, patients who are non-responders (i.e., patients who do not enroll in Quitline) will be randomized to receive either TM or continued-EHR. In phase three, patients in the TM condition who are non-responders will be randomized to receive either continued-TM or TM + phone coaching. DISCUSSION: This project will evaluate scalable, multi-level interventions to directly address strategic national priorities for reducing tobacco use and related disparities by increasing the Reach and Impact of evidence-based tobacco cessation interventions in low SES populations. TRIAL REGISTRATION: This trial was registered at ClinicalTrials.gov (NCT03900767) on April 4th, 2019.

Association of an Electronic Health Record Add-on App for Neonatal Bilirubin Management With Physician Efficiency and Care Quality.

Importance: The usefulness of electronic health record (EHR) systems could be significantly enhanced by innovative, third-party EHR add-on apps. Objective: To evaluate whether an EHR add-on app for neonatal bilirubin management can save clinicians time and improve patient care. Design, Setting, and Participants: This quality improvement study was conducted at the University of Utah Health Well Baby nursery and outpatient clinics and consisted of 4 substudies: (1) time savings were estimated in an experimental task-timing study comparing the time required for physicians to manage newborns' bilirubin levels with and without the add-on app, (2) app use was estimated from app logs, (3) health care use measures and guideline compliance were compared retrospectively before and after the intervention, and (4) clinician-perceived usability was measured through System Usability Scale surveys. The study took place between April 1, 2016, and September 3, 2019. Data analyses were conducted from October 30, 2018, to September 23, 2019. Interventions: At baseline, clinicians used a manual approach to ensure compliance with an evidence-based clinical guideline for neonatal bilirubin management. To facilitate guideline compliance, an EHR add-on app that automatically retrieves, organizes, and visualizes relevant patient data was developed. The app provides patient-specific assessments and recommendations, including the risk of rebound hyperbilirubinemia following phototherapy based on a predictive model. The add-on app was integrated with the University of Utah Health EHR on April 12, 2017. Main Outcomes and Measures: Clinician time savings, app use, health care use measures, guideline-compliant phototherapy ordering, and perceived usability as measured by the System Usability Scale survey. The survey is composed of 10 statements with responses ranging from 1 (strongly disagree) to 5 (strongly agree). The survey results in a single score ranging from 0 to 100, with ratings described as worst imaginable (mean System Usability Scale score, 12.5), awful (20.3), poor (35.7), okay (50.9), good (71.4), excellent (85.5), and best imaginable (90.9). Results: In 2018, the application was used 20 516 times by clinicians for 91.84% of eligible newborns. Use of the app saved 66 seconds for bilirubin management tasks compared with a commonly used tool (95% CI, 53-79 seconds; P < .001). Following the intervention, health care use rates remained stable, while orders for clinically appropriate phototherapy during hospitalization increased for newborns with bilirubin levels above the guideline-recommended threshold (odds ratio, 1.84; 95% CI, 1.16-2.90; P = .009). Surveys indicated excellent usability (System Usability Scale score, 83.90; 95% CI, 81.49-86.31). Conclusions and Relevance: Well-designed EHR add-on apps may save clinicians time and improve patient care. If time-saving apps, such as the bilirubin app, were implemented widely across institutions and care domains, the potential association with improved patient care and clinician efficiency could be significant. The University of Utah Health bilirubin app is being prepared for release into EHR app stores as free-to-use software.

Balancing Functionality versus Portability for SMART on FHIR Applications: Case Study for a Neonatal Bilirubin Management Application.

SMART on FHIR applications are standards-based tools integrated with electronic health record (EHR) systems and intended for dissemination across EHR platforms. A key challenge for disseminating many apps is that EHR vendors provide different levels of support for FHIR. Thus, app developers must balance functionality versus portability. In this case study, a feature-rich app for neonatal bilirubin management was developed prioritizing physician-requested functionality, with custom FHIR interfaces implemented within the EHR as needed. Following wide intra-institutional use, several approaches are being pursued for adapting the app for cross-institutional dissemination: user surveys and interviews to identify least-valued app features which could potentially be omitted; enabling the application to provide differential features depending on available EHR FHIR capabilities; replacing custom FHIR interfaces with native EHR FHIR interfaces as they became available; and using a canonical logical data model known as QUICK that can be mapped to different FHIR versions and profiles.

When an Alert is Not an Alert: A Pilot Study to Characterize Behavior and Cognition Associated with Medication Alerts.

Introduction. Preventable adverse drug events are a significant patient-safety concern, yet most medication alerts are disregarded. Pharmacists encounter the highest number of medication alerts and likely have developed behaviors to cope with alerting inefficiencies. The study objective was to better understand alert override behavior relating to a motivational construct framework. Methods. Mixed-methods study of 10 pharmacists (567 verifications) with eye-tracking observations and retrospective think aloud interviews. Results. Pharmacists spent on average 14 seconds longer verifying orders with alerts than orders without alerts (p<0.001). Verification occurred before alerts were triggered, and no order changes occurred after alerts. Pharmacists reported 62% of alerts as unhelpful and 21% as frustrating. Alert interactions took on average 3.9 seconds. Discussion. Pharmacists anticipate alerts by making appropriate checks and changes before alert prompts. Medication alerts seem to be useful. However, the observed pharmacists' behavior suggests changes in the alert context are needed to match cognition.

What are they trying to do?: An analysis of Action Identities in using electronic documentation in an EHR.

Documentation processes have changed substantially with EHR adoption. User satisfaction studies have focused on usability or cognitive analysis perspectives. Few studies have provided useful information to developers to improve designs. The purpose of this study is to report a 3-pronged approach to deepen understanding of the documentation process, with the intent to provide useful information for future design. This study was conducted in two phases, beginning with cognitive task interviews and observations, followed by post-observation interviews. Twenty-five constructs were identified across the phases, and we observed several patterns of note writing. Participants provided useful information to potentially inform future design. Our study illustrates how electronic documentation serves many clinical processes and is at the core of the medical record. Providers need multiple kinds of notes and ways to display notes. In order to meet provider goals, we must completely re-think the way electronic documentation is composed and displayed.

Checking the lists: A systematic review of electronic checklist use in health care.

OBJECTIVE: We conducted a literature search to examine the effects and experiences surrounding the transition from paper to electronic checklists in healthcare settings. We explore the types of electronic checklists being used in health care, how and where they were evaluated and seek to identify the successes and failures of using electronic checklists in healthcare, including use of checklists to ensure completeness of documentation in the electronic medical record. BACKGROUND: Formalized checklist use as a memory and decision aid in aviation has resulted in significant increases in safety in that domain. Checklists have also been successfully introduced to reduce errors in some areas of healthcare; however, in some contexts checklists failed to provide some of the expected benefits. Adapting and integrating checklists electronically into the healthcare workflow provides opportunities and challenges that need to be better understood to make checklist adoption a success in health care. METHOD: We conducted a literature search of the English language literature in MEDLINE using PubMed for peer-reviewed literature of implementation and use of electronic or computerized checklists related to clinical or healthcare use. We reviewed the studies and included in this review those papers that discussed in depth the development process and that conducted controlled studies to assess the effectiveness of checklists and the evaluation of their acceptance in the clinical context. RESULTS: The literature search using the keywords electronic checklist OR computerized checklist returned a total of 23 peer-reviewed papers. Out of these 15 were included in the review, with 8 excluded because they did not evaluate checklist use for patient care. APPLICATION: More rigorous application of known principles and methods from Human Computer Interaction research and the behavioral sciences can provide a clearer, more comprehensive understanding of the conditions that affect the development and use of checklists.

High Prevalence of Medication Discrepancies Between Home Health Referrals and Centers for Medicare and Medicaid Services Home Health Certification and Plan of Care and Their Potential to Affect Safety of Vulnerable Elderly Adults.

OBJECTIVES: To describe the prevalence of discrepancies between medication lists that referring providers and home healthcare (HH) nurses create. DESIGN: The active medication list from the hospital at time of HH initiation was compared with the HH agency's plan of care medication list. An electronic algorithm was developed to compare the two lists for discrepancies. SETTING: Single large hospital and HH agency in the western United States. PARTICIPANTS: Individuals referred for HH from the hospital in 2012 (N = 770, 96.3% male, median age 71). MEASUREMENTS: Prevalence was calculated for discrepancies, including medications missing from one list or the other and differences in dose, frequency, or route for medications contained on both lists. RESULTS: Participants had multiple medical problems (median 16 active problems) and were taking a median of 15 medications (range 1-93). Every participant had at least one discrepancy; 90.1% of HH lists were missing at least one medication that the referring provider had prescribed, 92.1% of HH lists contained medications not on the referring provider's list, 89.8% contained medication naming errors. 71.0% contained dosing discrepancies, and 76.3% contained frequency discrepancies. CONCLUSION: Discrepancies between HH and referring provider lists are common. Future work is needed to address possible safety and care coordination implications of discrepancies in this highly complex population.

Evaluation of an Electronic Module for Reconciling Medications in Home Health Plans of Care.

OBJECTIVES: Transitions in patient care pose an increased risk to patient safety. One way to reduce this risk is to ensure accurate medication reconciliation during the transition. Here we present an evaluation of an electronic medication reconciliation module we developed to reduce the transition risk in patients referred for home healthcare. METHODS: Nineteen physicians with experience in managing home health referrals were recruited to participate in this within-subjects experiment. Participants completed medication reconciliation for three clinical cases in each of two conditions. The first condition (paper-based) simulated current practice - reconciling medication discrepancies between a paper plan of care (CMS 485) and a simulated Electronic Health Record (EHR). For the second condition (electronic) participants used our medication reconciliation module, which we integrated into the simulated EHR. To evaluate the effectiveness of our medication reconciliation module, we employed repeated measures ANOVA to test the hypotheses that the module will: 1) Improve accuracy by reducing the number of unaddressed medication discrepancies, 2) Improve efficiency by reducing the reconciliation time, 3) have good perceived usability. RESULTS: The improved accuracy hypothesis is supported. Participants left more discrepancies unaddressed in the paper-based condition than the electronic condition, F (1,1) = 22.3, p < 0.0001 (Paper Mean = 1.55, SD = 1.20; Electronic Mean = 0.45, SD = 0.65). However, contrary to our efficiency hypothesis, participants took the same amount of time to complete cases in the two conditions, F (1, 1) =0.007, p = 0.93 (Paper Mean = 258.7 seconds, SD = 124.4; Electronic Mean = 260.4 seconds, SD = 158.9). The usability hypothesis is supported by a composite mean ability and confidence score of 6.41 on a 7-point scale, 17 of 19 participants preferring the electronic system and an SUS rating of 86.5. CONCLUSION: We present the evaluation of an electronic medication reconciliation module that increases detection and resolution of medication discrepancies compared to a paper-based process. Further work to integrate medication reconciliation within an electronic medical record is warranted.

Why aren't they happy? An analysis of end-user satisfaction with Electronic health records.

Introduction. Implementations of electronic health records (EHR) have been met with mixed outcome reviews. Complaints about these systems have led to many attempts to have useful measures of end-user satisfaction. However, most user satisfaction assessments do not focus on high-level reasoning, despite the complaints of many physicians. Our study attempts to identify some of these determinants. Method. We developed a user satisfaction survey instrument, based on pre-identified and important clinical and non-clinical clinician tasks. We surveyed a sample of in-patient physicians and focused on using exploratory factor analyses to identify underlying high-level cognitive tasks. We used the results to create unique, orthogonal variables representative of latent structure predictive of user satisfaction. Results. Our findings identified 3 latent high-level tasks that were associated with end-user satisfaction: a) High- level clinical reasoning b) Communicate/coordinate care and c) Follow the rules/compliance. Conclusion: We were able to successfully identify latent variables associated with satisfaction. Identification of communicability and high-level clinical reasoning as important factors determining user satisfaction can lead to development and design of more usable electronic health records with higher user satisfaction.

The influence of different graphical displays on nonexpert decision making under uncertainty.

Understanding how people interpret and use visually presented uncertainty data is an important yet seldom studied aspect of data visualization applications. Current approaches in visualization often display uncertainty as an additional data attribute without a well-defined context. Our goal was to test whether different graphical displays (glyphs) would influence a decision about which of 2 weather forecasts was a more accurate predictor of an uncertain temperature forecast value. We used a statistical inference task based on fictional univariate normal distributions, each characterized by a mean and standard deviation. Participants viewed 1 of 5 different glyph types representing 2 weather forecast distributions. Three of these used variations in spatial encoding to communicate the distributions and the other 2 used nonspatial encoding (brightness or color). Four distribution pairs were created with different relative standard deviations (uncertainty of the forecasts). We found that there was a difference in how decisions were made with spatial versus nonspatial glyphs, but no difference among the spatial glyphs themselves. Furthermore, the effect of different glyph types changed as a function of the variability of the distributions. The results are discussed in the context of how visualizations might improve decision making under uncertainty.

Epinome - a novel workbench for epidemic investigation and analysis of search strategies in public health practice.

We present a novel user-centric visual analytics system that supports investigation of simulated disease outbreak and the study of decision-making. We developed Epinome as part of our research on decision making in public health and in particular, on the evaluation of information search strategies in public health practice. Epinome is a highly dynamic web-based system that provides a platform to track and study subjects' decision making and information search strategies, under controlled and repeatable conditions using simulated disease outbreaks. In this paper we focus on the design and implementation of Epinome and present relevant results from field tests we conducted in Utah and Colorado.