Automating Electronic Health Record Data Quality Assessment.

Information systems such as Electronic Health Record (EHR) systems are susceptible to data quality (DQ) issues. Given the growing importance of EHR data, there is an increasing demand for strategies and tools to help ensure that available data are fit for use. However, developing reliable data quality assessment (DQA) tools necessary for guiding and evaluating improvement efforts has remained a fundamental challenge. This review examines the state of research on operationalising EHR DQA, mainly automated tooling, and highlights necessary considerations for future implementations. We reviewed 1841 articles from PubMed, Web of Science, and Scopus published between 2011 and 2021. 23 DQA programs deployed in real-world settings to assess EHR data quality (n = 14), and a few experimental prototypes (n = 9), were identified. Many of these programs investigate completeness (n = 15) and value conformance (n = 12) quality dimensions and are backed by knowledge items gathered from domain experts (n = 9), literature reviews and existing DQ measurements (n = 3). A few DQA programs also explore the feasibility of using data-driven techniques to assess EHR data quality automatically. Overall, the automation of EHR DQA is gaining traction, but current efforts are fragmented and not backed by relevant theory. Existing programs also vary in scope, type of data supported, and how measurements are sourced. There is a need to standardise programs for assessing EHR data quality, as current evidence suggests their quality may be unknown.

Vat Photopolymerization of Reinforced Styrene-Butadiene Elastomers: A Degradable Scaffold Approach.

Vat photopolymerization (VP) is a high-throughput additive manufacturing modality that also offers exceptional feature resolution and surface finish; however, the process is constrained by a limited selection of processable photocurable resins. Low resin viscosity (<10 Pa·s) is one of the most stringent process-induced constraints on resin processability, which in turn limits the mechanical performance of printed resin systems. Recently, the authors created a VP-processable photosensitive latex resin, where compartmentalization of the high molecular weight polymer chains into discrete particles resulted in the decoupling of viscosity from molecular weight. However, the monomers used to form the hydrogel green body resulted in decreased ultimate material properties due to the high cross-link density. Herein, we report a novel scaffold that allows for facile UV-based AM and simultaneously enhances the final part's material properties. This is achieved with a chemically labile acetal-containing cross-linker in conjunction with N-vinylpyrrolidone, which forms a glassy polymer after photocuring. Subsequent reactive extraction cleaves the cross-links and liberates the glassy polymer, which provides mechanical reinforcement of the geometrically complex VP-printed elastomer. With only a 0.1 wt % loading of photoinitiator, G'/G'' crossover times of less than 1 s and green body plateau moduli nearing 105 Pa are obtained. In addition, removal of the hydrophilic and thermally labile scaffold results in decreased water uptake and increased thermal stability of the final printed part. Ultimate strain and stress values of over 650% and 8.5 MPa, respectively, are achieved, setting a new benchmark for styrene-butadiene VP elastomers.

Managing Pandemic Responses with Health Informatics - Challenges for Assessing Digital Health Technologies.

OBJECTIVES: To highlight the role of technology assessment in the management of the COVID-19 pandemic. METHOD: An overview of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems. RESULTS: Evaluation of digital health technologies for COVID-19 should be based on their technical maturity as well as the scale of implementation. For mature technologies like telehealth whose efficacy has been previously demonstrated, pragmatic, rapid evaluation using the complex systems paradigm which accounts for multiple sociotechnical factors, might be more suitable to examine their effectiveness and emerging safety concerns in new settings. New technologies, particularly those intended for use on a large scale such as digital contract tracing, will require assessment of their usability as well as performance prior to deployment, after which evaluation should shift to using a complex systems paradigm to examine the value of information provided. The success of a digital health technology is dependent on the value of information it provides relative to the sociotechnical context of the setting where it is implemented. CONCLUSION: Commitment to evaluation using the evidence-based medicine and complex systems paradigms will be critical to ensuring safe and effective use of digital health technologies for COVID-19 and future pandemics. There is an inherent tension between evaluation and the imperative to urgently deploy solutions that needs to be negotiated.

3D Printing Latex: A Route to Complex Geometries of High Molecular Weight Polymers.

Vat photopolymerization (VP) additive manufacturing fabricates intricate geometries with excellent resolution; however, high molecular weight polymers are not amenable to VP due to concomitant high solution and melt viscosities. Thus, a challenging paradox arises between printability and mechanical performance. This report describes concurrent photopolymer and VP system design to navigate this paradox with the unprecedented use of polymeric colloids (latexes) that effectively decouple the dependency of viscosity on molecular weight. Photocrosslinking of a continuous-phase scaffold, which surrounds the latex particles, combined with in situ computer-vision print parameter optimization, which compensates for light scattering, enables high-resolution VP of high molecular weight polymer latexes as particle-embedded green bodies. Thermal post-processing promotes coalescence of the dispersed particles throughout the scaffold, forming a semi-interpenetrating polymer network without loss in part resolution. Printing a styrene-butadiene rubber latex, a previously inaccessible elastomer composition for VP, exemplified this approach and yielded printed elastomers with precise geometry and tensile extensibilities exceeding 500%.

Mixed methods protocol for a realist evaluation of electronic personal health records design features and use to support medication adherence (ePHRma).

BACKGROUND: National Health Service policy suggests that increasing usage of electronic personal health records (PHR) by patients will result in cost savings and improved public health. Medication adherence means that patients take their prescribed medication as agreed with their doctors. Some of the claimed benefits of PHRs are decreasing healthcare costs and improving medication adherence and patient outcomes. METHODS: This is a mixed methods convergent study, primarily qualitative. The qualitative and quantitative data collection and analysis will occur in parallel, and then be synthesised. We are interviewing and surveying adults with long-term conditions to identify what are the most important and useful features of their current PHR. The data collection comprises patient demographics, the Medication Adherence Questionnaire, the personality scale Big Five Inventory-2 Extra-Short Form and the WHO Quality of Life-BREF scale. Qualitative data will be analysed using the Framework method. ETHICS: We have received a favourable ethical opinion from the Health Research Authority/Research Ethics Committee.

Reflecting and Looking to the Future: What Is the Research Agenda for Theory in Health Informatics?

In this chapter, we reflect on the aim and objectives of the textbook and address known gaps in our theory coverage. We reinforce the importance of theory in health informatics and review the varying disciplinary origins of the theories considered in the book. We discuss the question of what makes a good theory and how to know which one is relevant for a given study. We recognize the limitations of the body of theory that we have presented and suggest what might be regarded as "native" theory that is original to health informatics. Finally, we propose topics to form a research agenda for theory in health informatics.

Artificial Intelligence in Clinical Decision Support: Challenges for Evaluating AI and Practical Implications.

OBJECTIVES: This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance. METHOD: A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems. RESULTS: There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed. CONCLUSION: Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application.

A review of measurement practice in studies of clinical decision support systems 1998-2017.

OBJECTIVE: To assess measurement practice in clinical decision support evaluation studies. MATERIALS AND METHODS: We identified empirical studies evaluating clinical decision support systems published from 1998 to 2017. We reviewed titles, abstracts, and full paper contents for evidence of attention to measurement validity, reliability, or reuse. We used Friedman and Wyatt's typology to categorize the studies. RESULTS: There were 391 studies that met the inclusion criteria. Study types in this cohort were primarily field user effect studies (n = 210) or problem impact studies (n = 150). Of those, 280 studies (72%) had no evidence of attention to measurement methodology, and 111 (28%) had some evidence with 33 (8%) offering validity evidence; 45 (12%) offering reliability evidence; and 61 (16%) reporting measurement artefact reuse. DISCUSSION: Only 5 studies offered validity assessment within the study. Valid measures were predominantly observed in problem impact studies with the majority of measures being clinical or patient reported outcomes with validity measured elsewhere. CONCLUSION: Measurement methodology is frequently ignored in empirical studies of clinical decision support systems and particularly so in field user effect studies. Authors may in fact be attending to measurement considerations and not reporting this or employing methods of unknown validity and reliability in their studies. In the latter case, reported study results may be biased and effect sizes misleading. We argue that replication studies to strengthen the evidence base require greater attention to measurement practice in health informatics research.

Team Competencies and Educational Threshold Concepts for Clinical Information Modelling.

Healthcare interoperability depends upon sound semantic models to support safe and reliable exchange of information. We argue that clinical information modelling requires a collaborative team of healthcare professionals, process and content analysts and terminologists and that 'separation of concerns' is unhelpful. We present six fundamental concepts that participants must understand to collaborate meaningfully in technology-agnostic information modelling.

The Safe and Effective Use of Shared Data Underpinned by Stakeholder Engagement and Evaluation Practice.

OBJECTIVES: The paper draws attention to: i) key considerations involving the confidentiality, privacy, and security of shared data; and ii) the requirements needed to build collaborative arrangements encompassing all stakeholders with the goal of ensuring safe, secure, and quality use of shared data. METHOD: A narrative review of existing research and policy approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Care and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems. RESULTS: The technological ability to merge, link, re-use, and exchange data has outpaced the establishment of policies, procedures, and processes to monitor the ethics and legality of shared use of data. Questions remain about how to guarantee the security of shared data, and how to establish and maintain public trust across large-scale shared data enterprises. This paper identifies the importance of data governance frameworks (incorporating engagement with all stakeholders) to underpin the management of the ethics and legality of shared data use. The paper also provides some key considerations for the establishment of national approaches and measures to monitor compliance with best practice. CONCLUSION: Data sharing endeavours can help to underpin new collaborative models of health care which provide shared information, engagement, and accountability amongst all stakeholders. We believe that commitment to rigorous evaluation and stakeholder engagement will be critical to delivering health data benefits and the establishment of collaborative models of health care into the future.

Genesis of a UK Faculty of Clinical Informatics at a time of anticipation for some, and ruby, golden and diamond celebrations for others.

This Editorial marks the launch of the UK Faculty of Clinical Informatics (FCI) at the time when non-clinically qualified informaticians are anticipating the lauch of  the Federation of Informatics Professionals in Health and Care (Fed-IP).

Informatics for Health 2017: Advancing both science and practice.

INTRODUCTION: The Informatics for Health congress, 24-26 April 2017, in Manchester, UK, brought together the Medical Informatics Europe (MIE) conference and the Farr Institute International Conference. This special issue of the Journal of Innovation in Health Informatics contains 113 presentation abstracts and 149 poster abstracts from the congress. DISCUSSION: The twin programmes of "Big Data" and "Digital Health" are not always joined up by coherent policy and investment priorities. Substantial global investment in health IT and data science has led to sound progress but highly variable outcomes. Society needs an approach that brings together the science and the practice of health informatics. The goal is multi-level Learning Health Systems that consume and intelligently act upon both patient data and organizational intervention outcomes. CONCLUSION: Informatics for Health demonstrated the art of the possible, seen in the breadth and depth of our contributions. We call upon policy makers, research funders and programme leaders to learn from this joined-up approach.

Measuring the operational impact of digitized hospital records: a mixed methods study.

BACKGROUND: Digitized (scanned) medical records have been seen as a means for hospitals to reduce costs and improve access to records. However, clinical usability of digitized records can potentially have negative effects on productivity. METHODS: Data were collected during follow-up outpatient consultations in two NHS hospitals by non-clinical observers using a work sampling approach in which pre-defined categories of clinician time usage were specified. Quantitative data was analysed using two-way ANOVA models and the Mann-Whitney U test. A focus group was held with clinicians to qualitatively explore their experiences using digitized medical records. The quantitative and qualitative results were synthesized. RESULTS: Four hundred six consultations were observed. Using paper records, there was a significant difference in consultation times between hospitals (p = 0.016) and a significant difference in consultation times between specialties within hospitals (p = 0.003). Using digitized records there was a significant difference in consultation times between specialties within a hospital (p = 0.001). Excluding outliers, there was no significant difference between consultation times using digitized records compared with consultations using paper records in the same hospital, either at site (p > =0.285) or specialty level (p > =0.122). With digitized records at site A, two out of three specialties showed a significant increase in time spent searching computer records (p < =0.010, Δ = 01:50-07:10) and one specialty had a corresponding reduction in time spent searching paper records (p = 0.015, Δ = -00:28). Site B showed a notable increase in direct patient care (p < 0.001, Δ = 04:20-06:00) and time spent searching computer records (p < =0.043, Δ = 00:10-01:40) and reductions in the other time categories. The focus group confirmed that the most recent clinical letter was a vital document in the patient record, often containing most of the required information. Concerns were expressed about consistency of scanning practice, causing uncertainty about what could be relied upon to exist in the digitized record. Benefits of digitized records included: access from multiple locations, better prepared ward rounds, improved inpatient handovers and an improved timeline of patient events. Limitations of digitized records included: increased complexity of creating a patient summary, display of specialised content such as hand-drawn diagrams, inability to quickly flick through the pages to find relevant content. CONCLUSIONS: Digitized medical records can be implemented without detrimental operational impact. Inherent differences between specialties can outweigh the differences between paper and digitized records. Clear and consistent operational processes are vital for the reliability and usability of digitized medical records. Divergent views about usability (such as whether patient summary information is better or worse) may reflect familiarity with features of the digitized record.

Theoretical Foundations for Evidence-Based Health Informatics: Why? How?

A scientific approach to health informatics requires sound theoretical foundations. Health informatics implementation would be more effective if evidence-based and guided by theories about what is likely to work in what circumstances. We report on a Medinfo 2015 workshop on this topic jointly organized by the EFMI Working Group on Assessment of Health Information Systems and the IMIA Working Group on Technology Assessment and Quality Development. We discuss the findings of the workshop and propose an approach to consolidate empirical knowledge into testable middle-range theories.

Mixed Methods: A Paradigm for Holistic Evaluation of Health IT.

This contribution offers an overview of the 'third research paradigm', its historical roots and its relevance for health informatics. Using illustrative studies, we explore the concepts of triangulation and integration of quantitative and qualitative data and refute common philosophical objections to mixing different types of knowledge. We consider how the mixed method paradigm relates to two programme design and evaluation frameworks that are important for health informatics: realist evaluation and Theory of Change. We discuss how to manage practical challenges to this approach and explain how mixed method studies support an evidence-based approach to real world policy, planning and investment decisions.

STAT-HI: A socio-technical assessment tool for health informatics implementations.

This paper proposes a socio-technical assessment tool (STAT-HI) for health informatics implementations. We explore why even projects allegedly using sound methodologies repeatedly fail to give adequate attention to socio-technical issues, and we present an initial draft of a structured assessment tool for health informatics implementation that encapsulates socio-technical good practice. Further work is proposed to enrich and validate the proposed instrument. This proposal was presented for discussion at a meeting of the UK Faculty of Health Informatics in December 2009.

Developing a theoretical model of clinician information usage propensity.

Based on qualitative research, we developed the theoretical construct "clinician information usage propensity" as a hypothetical indicator of attitudes and behaviour towards clinical information and systems. We devised a survey to validate the construct and had 146 responses. Principal components analysis extracted four factors accounting for 47.2% of the variance: beliefs about clinical judgement, beliefs about information quality, cultural resistance and cognitive approach. The components were reasonably consistent with the model but two factors (beliefs about information quality, cognitive approach) had low reliability (alpha<0.6). Cultural resistance was the main factor and correlated with gender, grade and age group. Female clinicians showed significantly higher cultural resistance and preference for narrative; hospital doctors generally had higher cultural resistance than general practitioners. As only 47.2% of the variance was explained, further work is needed to refine the instrument to remove redundancy, improve sensitivity on the identified components and allow the construct to be explored as a form of technology adoption model. We posit that beliefs about clinical judgement merit further attention in medical informatics research.