Are We There Yet? Human Factors Knowledge and Health Information Technology - the Challenges of Implementation and Impact.

Objective: To review the developments in human factors (HF) research on the challenges of health information technology (HIT) implementation and impact given the continuing incidence of usability problems and unintended consequences from HIT development and use. Methods: A search of PubMed/Medline and Web of Science® identified HF research published in 2015 and 2016. Electronic health records (EHRs) and patient-centred HIT emerged as significant foci of recent HF research. The authors selected prominent papers highlighting ongoing HF and usability challenges in these areas. This selective rather than systematic review of recent HF research highlights these key challenges and reflects on their implications on the future impact of HF research on HIT. Results: Research provides evidence of continued poor design, implementation, and usability of HIT, as well as technology-induced errors and unintended consequences. The paper highlights support for: (i) strengthening the evidence base on the benefits of HF approaches; (ii) improving knowledge translation in the implementation of HF approaches during HIT design, implementation, and evaluation; (iii) increasing transparency, governance, and enforcement of HF best practices at all stages of the HIT system development life cycle. Discussion and Conclusion: HF and usability approaches are yet to become embedded as integral components of HIT development, implementation, and impact assessment. As HIT becomes ever-more pervasive including with patients as end-users, there is a need to expand our conceptualisation of the problems to be addressed and the suite of tactics and strategies to be used to calibrate our pro-active involvement in its improvement.

Methods for Addressing Technology-induced Errors: The Current State.

OBJECTIVES: The objectives of this paper are to review and discuss the methods that are being used internationally to report on, mitigate, and eliminate technology-induced errors. METHODS: The IMIA Working Group for Health Informatics for Patient Safety worked together to review and synthesize some of the main methods and approaches associated with technology- induced error reporting, reduction, and mitigation. The work involved a review of the evidence-based literature as well as guideline publications specific to health informatics. RESULTS: The paper presents a rich overview of current approaches, issues, and methods associated with: (1) safe HIT design, (2) safe HIT implementation, (3) reporting on technology-induced errors, (4) technology-induced error analysis, and (5) health information technology (HIT) risk management. The work is based on research from around the world. CONCLUSIONS: Internationally, researchers have been developing methods that can be used to identify, report on, mitigate, and eliminate technology-induced errors. Although there remain issues and challenges associated with the methodologies, they have been shown to improve the quality and safety of HIT. Since the first publications documenting technology-induced errors in healthcare in 2005, we have seen in a short 10 years researchers develop ways of identifying and addressing these types of errors. We have also seen organizations begin to use these approaches. Knowledge has been translated into practice in a short ten years whereas the norm for other research areas is of 20 years.

Human Factors for More Usable and Safer Health Information Technology: Where Are We Now and Where do We Go from Here?

A wide range of human factors approaches have been developed and adapted to healthcare for detecting and mitigating negative unexpected consequences associated with technology in healthcare (i.e. technology-induced errors). However, greater knowledge and wider dissemination of human factors methods is needed to ensure more usable and safer health information technology (IT) systems. OBJECTIVE: This paper reports on work done by the IMIA Human Factors Working Group and discusses some successful approaches that have been applied in using human factors to mitigate negative unintended consequences of health IT. The paper addresses challenges in bringing human factors approaches into mainstream health IT development. RESULTS: A framework for bringing human factors into the improvement of health IT is described that involves a multi-layered systematic approach to detecting technology-induced errors at all stages of a IT system development life cycle (SDLC). Such an approach has been shown to be needed and can lead to reduced risks associated with the release of health IT systems into live use with mitigation of risks of negative unintended consequences. CONCLUSION: Negative unintended consequences of the introduction of IT into healthcare (i.e. potential for technology-induced errors) continue to be reported. It is concluded that methods and approaches from the human factors and usability engineering literatures need to be more widely applied, both in the vendor community and in local and regional hospital and healthcare settings. This will require greater efforts at dissemination and knowledge translation, as well as greater interaction between the academic and vendor communities.

Patient-Centred Coordinated Care in Times of Emerging Diseases and Epidemics. Contribution of the IMIA Working Group on Patient Safety.

OBJECTIVES: In this paper the researchers describe how existing health information technologies (HIT) can be repurposed and new technologies can be innovated to provide patient-centered care to individuals affected by new and emerging diseases. METHODS: The researchers conducted a focused review of the published literature describing how HIT can be used to support safe, patient-centred, coordinated care to patients who are affected by Ebola (an emerging disease). RESULTS: New and emerging diseases present opportunities for repurposing existing technologies and for stimulating the development of new HIT innovation. Innovative technologies may be developed such as new software used for tracking patients during new or emerging disease outbreaks or by repurposing and extending existing technologies so they can be used to support patients, families and health professionals who may have been exposed to a disease. The paper describes the development of new technologies and the repurposing and extension of existing ones (such as electronic health records) using the most recent outbreak of Ebola as an example.

Human Factors in the Large: Experiences from Denmark, Finland and Canada in Moving Towards Regional and National Evaluations of Health Information System Usability. Contribution of the IMIA Human Factors Working Group.

OBJECTIVES: The objective of this paper is to explore approaches to understanding the usability of health information systems at regional and national levels. METHODS: Several different methods are discussed in case studies from Denmark, Finland and Canada. They range from small scale qualitative studies involving usability testing of systems to larger scale national level questionnaire studies aimed at assessing the use and usability of health information systems by entire groups of health professionals. RESULTS: It was found that regional and national usability studies can complement smaller scale usability studies, and that they are needed in order to understand larger trends regarding system usability. Despite adoption of EHRs, many health professionals rate the usability of the systems as low. A range of usability issues have been noted when data is collected on a large scale through use of widely distributed questionnaires and websites designed to monitor user perceptions of usability. CONCLUSION: As health information systems are deployed on a widespread basis, studies that examine systems used regionally or nationally are required. In addition, collection of large scale data on the usability of specific IT products is needed in order to complement smaller scale studies of specific systems.

A methodological approach for designing a usable ontology-based GUI in healthcare.

This paper presents a methodological approach to the design and evaluation of an interface for an ontology-based system used for designing care plans for monitoring patients at home. In order to define the care plans, physicians need a tool for creating instances of the ontology and configuring some rules. Our purpose is to develop an interface to allow clinicians to interact with the ontology. Although ontology-driven applications do not necessarily present the ontology in the user interface, it is our hypothesis that showing selected parts of the ontology in a "usable" way could enhance clinician's understanding and make easier the definition of the care plans. Based on prototyping and iterative testing, this methodology combines visualization techniques and usability methods. Preliminary results obtained after a formative evaluation indicate the effectiveness of suggested combination.

Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches. Contribution of the IMIA Working Group Health Informatics for Patient Safety.

OBJECTIVES: Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence-based approaches to applying usability engineering methods. METHODS: A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. RESULTS: The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. CONCLUSION: The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

From Usability Testing to Clinical Simulations: Bringing Context into the Design and Evaluation of Usable and Safe Health Information Technologies. Contribution of the IMIA Human Factors Engineering for Healthcare Informatics Working Group.

OBJECTIVES: The objective of this paper is to explore human factors approaches to understanding the use of health information technology (HIT) by extending usability engineering approaches to include analysis of the impact of clinical context through use of clinical simulations. METHODS: Methods discussed are considered on a continuum from traditional laboratory-based usability testing to clinical simulations. Clinical simulations can be conducted in a simulation laboratory and they can also be conducted in real-world settings. The clinical simulation approach attempts to bring the dimension of clinical context into stronger focus. This involves testing of systems with representative users doing representative tasks, in representative settings/environments. RESULTS: Application of methods where realistic clinical scenarios are used to drive the study of users interacting with systems under realistic conditions and settings can lead to identification of problems and issues with systems that may not be detected using traditional usability engineering methods. In conducting such studies, careful consideration is needed in creating ecologically valid test scenarios. The evidence obtained from such evaluation can be used to improve both the usability and safety of HIT. In addition, recent work has shown that clinical simulations, in particular those conducted in-situ, can lead to considerable benefits when compared to the costs of running such studies. CONCLUSION: In order to bring context of use into the testing of HIT, clinical simulation, involving observing representative users carrying out tasks in representative settings, holds considerable promise.

Discussion of "Attitude of physicians towards automatic alerting in computerized physician order entry systems".

With these comments on the paper "Attitude of Physicians Towards Automatic Alerting in Computerized Physician Order Entry Systems", written by Martin Jung and co-authors, with Dr. Elske Ammenwerth as senior author [1], the journal wants to stimulate a broad discussion on computerized physician order entry systems. An international group of experts have been invited by the editor of Methods to comment on this paper. Each of the invited commentaries forms one section of this paper.

Empowering Patients: Making Health Information and Systems Safer for Patients and the Public. Contribution of the IMIA Health Informatics for Patient Safety Working Group.

OBJECTIVES: The objectives of this paper are to explore issues and perspectives from four regions of the world where health information systems are contributing to patient empowerment and influencing patient safety. METHODS: Members of the IMIA Working Group for Health Information Systems Safety came together to explore global issues at the intersection of health information systems safety, patient empowerment and patient safety. The group carried out a review and synthesis of the empirical and grey literature in four different regions/countries of the world that have differing health information system safety priorities. RESULTS: Regions/countries from differing parts of the world are developing: (1) high quality, safe information for individuals to use in their health related decision making, (2) patient portals and testing them for their safety, (3) methods for identifying unsafe health information system features and functions, and (4) ways of engaging citizens in identifying unsafe features and functions of health information systems. CONCLUSIONS: Internationally, there has been a rise in the number of health information systems and technologies that are being developed to support patient care. The amount of health information available on the World Wide Web (WWW), and the use of mobile phone software to support consumer health behaviours and self-management of chronic illnesses has also grown. The use of some of these health information systems and technologies has helped citizens to improve their health status (e.g. patient portals, mobile phones). However, the safety of these systems and technologies has come into question. As a result, there is a need to refine these systems and ensure their safety when they are used by patients and their families.

Technology-induced errors. The current use of frameworks and models from the biomedical and life sciences literatures.

OBJECTIVE: The objective of this paper is to examine the extent, range and scope to which frameworks, models and theories dealing with technology-induced error have arisen in the biomedical and life sciences literature as indexed by Medline®. METHODS: To better understand the state of work in the area of technology-induced error involving frameworks, models and theories, the authors conducted a search of Medline® using selected key words identified from seminal articles in this research area. Articles were reviewed and those pertaining to frameworks, models or theories dealing with technology-induced error were further reviewed by two researchers. RESULTS: All articles from Medline® from its inception to April of 2011 were searched using the above outlined strategy. 239 citations were returned. Each of the abstracts for the 239 citations were reviewed by two researchers. Eleven articles met the criteria based on abstract review. These 11 articles were downloaded for further in-depth review. The majority of the articles obtained describe frameworks and models with reference to theories developed in other literatures outside of healthcare. The papers were grouped into several areas. It was found that articles drew mainly from three literatures: 1) the human factors literature (including human-computer interaction and cognition), 2) the organizational behavior/sociotechnical literature, and 3) the software engineering literature. CONCLUSIONS: A variety of frameworks and models were found in the biomedical and life sciences literatures. These frameworks and models drew upon and extended frameworks, models and theoretical perspectives that have emerged in other literatures. These frameworks and models are informing an emerging line of research in health and biomedical informatics involving technology-induced errors in healthcare.

Reporting observational studies of the use of information technology in the clinical consultation. A position statement from the IMIA Primary Health Care Informatics Working Group (IMIA PCI WG).

OBJECTIVES: To develop a classification system to improve the reporting of observational studies of the use of information technology (IT) in clinical consultations. METHODS: Literature review, workshops, and development of a position statement. We grouped the important aspects for consistent reporting into a "faceted classification"; the components relevant to a particular study to be used independently. RESULTS: The eight facets of our classification are: (1) Theoretical and methodological approach: e.g. dramaturgical, cognitive; (2) DATA COLLECTION: Type and method of observation; (3) Room layout and environment: How this affects interaction between clinician, patient and computer. (4) Initiation and Interaction: Who starts the consultation, and how the participants interact; (5) Information and knowledge utilisation: What sources of information or decision support are used or provided; (6) Timing and type of consultation variables: Standard descriptors that can be used to allow comparison of duration and description of continuous activities (e.g. speech, eye contact) and episodic ones, such as prescribing; (7) Post-consultation impact measures: Satisfaction surveys and health economic assessment based on the perceived quality of the clinician-patient interaction; and (8) Data capture, storage, and export formats: How to archive and curate data to facilitate further analysis. CONCLUSIONS: Adoption of this classification should make it easier to interpret research findings and facilitate the synthesis of evidence across studies. Those engaged in IT-consultation research shouldconsider adopting this reporting guide.

Towards an integrative cognitive-socio-technical approach in health informatics: analyzing technology-induced error involving health information systems to improve patient safety.

The purpose of this paper is to argue for an integration of cognitive and socio-technical approaches to assessing the impact of health information systems. Historically, health informatics research has examined the cognitive and socio-technical aspects of health information systems separately. In this paper we argue that evaluations of health information systems should consider aspects related to cognition as well as socio-technical aspects including impact on workflow (i.e. an integrated view). Using examples from the study of technology-induced error in healthcare, we argue for the use of simulations to evaluate the cognitive-socio-technical impacts of health information technology [36]. Implications of clinical simulations and analysis of cognitive-social-technical impacts are discussed within the context of the system development life cycle to improve health information system design, implementation and evaluation.

Application of the Apriori algorithm for adverse drug reaction detection.

The objective of this research is to assess the suitability of the Apriori association analysis algorithm for the detection of adverse drug reactions (ADR) in health care data. The Apriori algorithm is used to perform association analysis on the characteristics of patients, the drugs they are taking, their primary diagnosis, co-morbid conditions, and the ADRs or adverse events (AE) they experience. This analysis produces association rules that indicate what combinations of medications and patient characteristics lead to ADRs. A simple data set is used to demonstrate the feasibility and effectiveness of the algorithm.

A framework for diagnosing and identifying where technology-induced errors come from.

Health information systems have the ability to reduce medical errors but they can also introduce new types of errors. In the cognitive and human factors literature there is a recognition that many of the high profile accidents that have occurred in other industries outside of healthcare have had their origins in the complexities of organizational work and how work is structured. The authors propose that in order to have a fully robust framework for diagnosing technology-induced errors one must understand the development and implementation of a technology and the influences of policy using a multi-organizational model. The authors propose that technology-induced errors may have their origins in up to four or more organizational structures that make up complex health care systems in addition to the health care provider: governments, model organizations, software development organizations, and local healthcare organizations. In this paper a framework for considering the origins of technology-induced error in healthcare is presented, along with our experiences to date in the application of the framework.

Using a low-cost simulation approach for assessing the impact of a medication administration system on workflow.

This paper describes the analysis of the impact of a medication administration system on clinical workflow. The methodological framework employed was based on in-depth analysis of simulated user interactions with a medication administration system. The approach involved the collection of rich data consisting of audio and video recordings of interactions between 16 subjects (5 nurses and 11 physicians) as they interacted with a medication administration system. Methodological considerations and issues in conducting such studies are discussed. The study indicated that use of the system would have a significant impact on nurse and physician workflow and that this impact could be accurately identified using simulation approaches prior to widespread release of such systems in real clinical environments.

The School of Health Information Science at the University of Victoria: towards an intergrative model for health informatics education and research.

OBJECTIVES: There is an increasing need for well qualified health informatics practitioners and for educational programs that produce them. Since 1981, the School of Health Information Science at the University of Victoria has delivered a range of educational programs in health informatics. The School's objective has been to produce graduates who can assume a range of roles in health informatics, including managers, developers, researchers and evaluators of health care systems. METHODS: The approach taken by the School has been to provide an integrated 'holistic' approach to health informatics education that balances both theory and practice. The curriculum has emphasized interdisciplinary skills and has been based on a process of consultation with key stakeholders in both industry and academia. In addition, several new distance collaborative models for health informatics education (including a distributed MSc degree program) have been recently initiated through the University of Victoria with collaborating Canadian universities. RESULTS: To date, graduates of the programs offered have become highly sought after, with the demand for graduates of the programs continually exceeding the number of graduates. The core undergraduate curriculum has recently been undergone refinement to include training in new emerging areas of health informatics. In addition, a distributed MSc program has been successfully initiated by the School, currently with 23 students participating from dispersed geographical locations across Canada. CONCLUSIONS: The School of Health Information Science at the University of Victoria has been involved in providing unique interdisciplinary education in health informatics for over twenty years. The School continues to maintain its emphasis on integrated education, refining its curriculum and moving into new areas such as distance education and cross-Canadian collaborations.

Reflections on an arranged marriage between bioinformatics and health informatics.

OBJECTIVE: To compare the discussions of two workshops held during 2001 by two Canadian organisations, HEALNet, a Network of Centres of Excellence for research in health information applications, and Genome Canada, a national research funding agency for genomics and proteomics, in collaboration with the Institute of Genetics of the Canadian Institutes of Health Research, to examine strategic research development in Health Informatics and Bioinformatics respectively. METHODS: Invited workshops with structured debate. Concept analysis of preparative material and debates. RESULTS: A predominantly common set of concepts was discerned from both workshops. Analysis of published definitions showed an inability to distinguish a definition that would suggest that health informatics and bioinformatics are separate disciplines. In both workshops there was evidence of deep concerns of identity, the lack of clear structures to support research funding as well as uncertainty in distinguishing between service and research. CONCLUSIONS: Many deep issues currently inhibit the recognition and funding of research in health and bioinformatics in Canada and elsewhere. Some of these issues are common to both health and bioinformatics. The overlap in prevailing definitions, research concerns and methodological content in the respective domains suggest that common research needs should be better identified and reinforced for the benefit of both.

What do patients do with access to their medical records?

We sought to study the phenomenon of patients having access to their own medical records in order to determine the impact on them and on their relationship with their health care providers. We created the Patient Clinical Information System (PatCIS) to interface with the clinical data repository at New York Presbyterian Hospital to allow patients to add to and review their medical data. We also provided educational resources and automated advice programs. We provided access to the system to thirteen subjects over a nineteen-month period and reviewed their activities in the system's usage log. We also collected data via questionnaire and telephone interview. We found that patients varied in their use of the system, from once a month or less to one or more times per day. All patients primarily used the system to review laboratory results. Both they and their physicians believed that use of the system enhanced the patients' understanding of their conditions and improved their communication with their physicians. There were no adverse events encountered during the study.

'Televaluation' of clinical information systems: an integrative approach to assessing Web-based systems.

The World Wide Web provides an unprecedented opportunity for widespread access to health-care applications by both patients and providers. The development of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance evaluation (i.e. 'televaluation') of emerging Web-based information technologies. In health informatics evaluation, there is a need for application of new ideas and methods from the fields of cognitive science and usability engineering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video-based usability testing of user interactions with clinical information systems. Examples illustrating application of these techniques are presented for the assessment of a patient clinical information system (PatCIS), as well as an evaluation of use of Web-based clinical guidelines. Issues in designing, prototyping and iteratively refining evaluation components are discussed, along with description of a 'virtual' usability laboratory.