AI Quality Standards in Health Care: Rapid Umbrella Review.

BACKGROUND: In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies. OBJECTIVE: This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period. METHODS: We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies. RESULTS: We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard-related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications. CONCLUSIONS: Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

Humanizing Big Data and Detailing Social Determinants of Health via Information Visualizations.

The pandemic has had devastating impacts on humanity and the global healthcare sector. An analysis into the social determinants of health, in particular racial and ethnic disparities may explain why certain population groups have been disproportionately affected by COVID-19. The objective of this study is to humanize and personify numerical data. Additionally, COVID-19 population data will be stratified via three data visualization tools (i.e., a persona, a journey map, Sankey diagram) to create a Visualized Combined Experience (VCE) Diagram to illustrate the micro, and macro, perspectives of marginalized individuals across the continuum of care.

Modelling Information Needs and Sources in a COVID-19 Designated Hospital.

COVID-19 remains an important focus of study in the field of public health informatics. COVID-19 designated hospitals have played an important role in the management of patients affected by the disease. In this paper we describe our modelling of the needs and sources of information for infectious disease practitioners and hospital administrators used to manage a COVID-19 outbreak. Infectious disease practitioner and hospital administrator stakeholders were interviewed to learn about their information needs and where they obtained their information. Stakeholder interview data were transcribed and coded to extract use case information. The findings indicate that participants used many and varied sources of information in the management of COVID-19. The use of multiple, differing sources of data led to considerable effort. In modelling participants' activities, we identified potential subsystems that could be used as a basis for developing an information system specific to the public health needs of hospitals providing care to COVID-19 patients.

Towards the Adoption of Novel Visualizations in Public Health.

Visualizations form an important part of public health informatics (PHI) communications. Visualizing data facilitates discussion, aids understanding, makes patterns apparent, promotes analysis, and fosters recall. How rare are novel visualizations in the PHI literature? In Phase 1, we used a rapid review methodology to test the commonness of the Sankey diagram in the PHI theory literature via an automated text search for key terms. In Phase 2, we prototype an uncommon chart type. A total of 27 relvant papers were searched and a computer-generated Sankey diagram was prototyped. PHI professionals have access to visualization tools emerging from social media and niche systems. PHI literature underutilizes uncommon visualizations requiring programming expertise. The authors advocate for: multi-disciplinary teamwork, technical education, the use of open visualization tools, and further adoption of visualization for public health professionals.

Factors Influencing Clinician Trust in Predictive Clinical Decision Support Systems for In-Hospital Deterioration: Qualitative Descriptive Study.

BACKGROUND: Clinician trust in machine learning-based clinical decision support systems (CDSSs) for predicting in-hospital deterioration (a type of predictive CDSS) is essential for adoption. Evidence shows that clinician trust in predictive CDSSs is influenced by perceived understandability and perceived accuracy. OBJECTIVE: The aim of this study was to explore the phenomenon of clinician trust in predictive CDSSs for in-hospital deterioration by confirming and characterizing factors known to influence trust (understandability and accuracy), uncovering and describing other influencing factors, and comparing nurses' and prescribing providers' trust in predictive CDSSs. METHODS: We followed a qualitative descriptive methodology conducting directed deductive and inductive content analysis of interview data. Directed deductive analyses were guided by the human-computer trust conceptual framework. Semistructured interviews were conducted with nurses and prescribing providers (physicians, physician assistants, or nurse practitioners) working with a predictive CDSS at 2 hospitals in Mass General Brigham. RESULTS: A total of 17 clinicians were interviewed. Concepts from the human-computer trust conceptual framework-perceived understandability and perceived technical competence (ie, perceived accuracy)-were found to influence clinician trust in predictive CDSSs for in-hospital deterioration. The concordance between clinicians' impressions of patients' clinical status and system predictions influenced clinicians' perceptions of system accuracy. Understandability was influenced by system explanations, both global and local, as well as training. In total, 3 additional themes emerged from the inductive analysis. The first, perceived actionability, captured the variation in clinicians' desires for predictive CDSSs to recommend a discrete action. The second, evidence, described the importance of both macro- (scientific) and micro- (anecdotal) evidence for fostering trust. The final theme, equitability, described fairness in system predictions. The findings were largely similar between nurses and prescribing providers. CONCLUSIONS: Although there is a perceived trade-off between machine learning-based CDSS accuracy and understandability, our findings confirm that both are important for fostering clinician trust in predictive CDSSs for in-hospital deterioration. We found that reliance on the predictive CDSS in the clinical workflow may influence clinicians' requirements for trust. Future research should explore the impact of reliance, the optimal explanation design for enhancing understandability, and the role of perceived actionability in driving trust.

Opportunities to Improve COVID-19 Dashboard Designs for the Public.

Many organizations created COVID-19 dashboards to communicate epidemiologic statistics or community health capabilities with the public. In this paper we used dashboard heuristics to identify common violations observed in COVID-19 dashboards targeted to citizens. Many of the faults we identified likely stem from failing to include users in the design of these dashboards. We urge health information dashboard designers to implement design principles and test dashboards with representative users to ensure that their tools are satisfying user needs.

A review of healthcare information system usability & safety.

Healthcare information systems have been designed to increase the efficiency and safety of healthcare processes. Systems such as electronic health records and pervasive computing devices have been shown to improve the safety of healthcare. However, increasing research has indicated that the design of such systems, in particular the user interface, may be related to increased incidence of other types of error. In this review, the relationship between human factors and usability will be considered in the context of designing safe and effective healthcare applications, with a focus on hand-held computing devices. Medline was searched for the specific terms listed below and restricted to the date ranges 2006-01-01 through to 2011-03-03: (error AND technology AND human factors); (error AND (CPOE OR (Computerized AND provider AND order AND entry))); (Technology AND Induced AND Error). The returned list of papers was screened by examining titles and abstracts to select candidate papers for further review. The initial search yield was 239 papers. On reviewing the title and abstract, 186 were rejected and 51 papers remained for analysis. New technology, such as CPOE, offers improvements over traditional paper tools and it is shown to have a positive effect on patient safety. New technology also creates the opportunity for new errors to occur and lead to the coining of the term "technology-induced error". The magnitude of the usability-testing needs is larger than it may seem.