Machine Learning Model for Readmission Prediction of Patients With Heart Failure Based on Electronic Health Records: Protocol for a Quasi-Experimental Study for Impact Assessment.

BACKGROUND: Care for patients with heart failure (HF) causes a substantial load on health care systems where a prominent challenge is the elevated rate of readmissions within 30 days following initial discharge. Clinical professionals face high levels of uncertainty and subjectivity in the decision-making process on the optimal timing of discharge. Unwanted hospital stays generate costs and cause stress to patients and potentially have an impact on care outcomes. Recent studies have aimed to mitigate the uncertainty by developing and testing risk assessment tools and predictive models to identify patients at risk of readmission, often using novel methods such as machine learning (ML). OBJECTIVE: This study aims to investigate how a developed clinical decision support (CDS) tool alters the decision-making processes of health care professionals in the specific context of discharging patients with HF, and if so, in which ways. Additionally, the aim is to capture the experiences of health care practitioners as they engage with the system's outputs to analyze usability aspects and obtain insights related to future implementation. METHODS: A quasi-experimental design with randomized crossover assessment will be conducted with health care professionals on HF patients' scenarios in a region located in the South of Sweden. In total, 12 physicians and nurses will be randomized into control and test groups. The groups shall be provided with 20 scenarios of purposefully sampled patients. The clinicians will be asked to take decisions on the next action regarding a patient. The test group will be provided with the 10 scenarios containing patient data from electronic health records and an outcome from an ML-based CDS model on the risk level for readmission of the same patients. The control group will have 10 other scenarios without the CDS model output and containing only the patients' data from electronic medical records. The groups will switch roles for the next 10 scenarios. This study will collect data through interviews and observations. The key outcome measures are decision consistency, decision quality, work efficiency, perceived benefits of using the CDS model, reliability, validity, and confidence in the CDS model outcome, integrability in the routine workflow, ease of use, and intention to use. This study will be carried out in collaboration with Cambio Healthcare Systems. RESULTS: The project is part of the Center for Applied Intelligent Systems Research Health research profile, funded by the Knowledge Foundation (2021-2028). Ethical approval for this study was granted by the Swedish ethical review authority (2022-07287-02). The recruitment process of the clinicians and the patient scenario selection will start in September 2023 and last till March 2024. CONCLUSIONS: This study protocol will contribute to the development of future formative evaluation studies to test ML models with clinical professionals. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/52744.

A Framework to Guide Implementation of AI in Health Care: Protocol for a Cocreation Research Project.

BACKGROUND: Artificial intelligence (AI) has the potential in health care to transform patient care and administrative processes, yet health care has been slow to adopt AI due to many types of barriers. Implementation science has shown the importance of structured implementation processes to overcome implementation barriers. However, there is a lack of knowledge and tools to guide such processes when implementing AI-based applications in health care. OBJECTIVE: The aim of this protocol is to describe the development, testing, and evaluation of a framework, "Artificial Intelligence-Quality Implementation Framework" (AI-QIF), intended to guide decisions and activities related to the implementation of various AI-based applications in health care. METHODS: The paper outlines the development of an AI implementation framework for broad use in health care based on the Quality Implementation Framework (QIF). QIF is a process model developed in implementation science. The model guides the user to consider implementation-related issues in a step-by-step design and plan and perform activities that support implementation. This framework was chosen for its adaptability, usability, broad scope, and detailed guidance concerning important activities and considerations for successful implementation. The development will proceed in 5 phases with primarily qualitative methods being used. The process starts with phase I, in which an AI-adapted version of QIF is created (AI-QIF). Phase II will produce a digital mockup of the AI-QIF. Phase III will involve the development of a prototype of the AI-QIF with an intuitive user interface. Phase IV is dedicated to usability testing of the prototype in health care environments. Phase V will focus on evaluating the usability and effectiveness of the AI-QIF. Cocreation is a guiding principle for the project and is an important aspect in 4 of the 5 development phases. The cocreation process will enable the use of both on research-based and practice-based knowledge. RESULTS: The project is being conducted within the frame of a larger research program, with the overall objective of developing theoretically and empirically informed frameworks to support AI implementation in routine health care. The program was launched in 2021 and has carried out numerous research activities. The development of AI-QIF as a tool to guide the implementation of AI-based applications in health care will draw on knowledge and experience acquired from these activities. The framework is being developed over 2 years, from January 2023 to December 2024. It is under continuous development and refinement. CONCLUSIONS: The development of the AI implementation framework, AI-QIF, described in this study protocol aims to facilitate the implementation of AI-based applications in health care based on the premise that implementation processes benefit from being well-prepared and structured. The framework will be coproduced to enhance its relevance, validity, usefulness, and potential value for application in practice. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/50216.

Barriers and Enablers for Implementation of an Artificial Intelligence-Based Decision Support Tool to Reduce the Risk of Readmission of Patients With Heart Failure: Stakeholder Interviews.

BACKGROUND: Artificial intelligence (AI) applications in health care are expected to provide value for health care organizations, professionals, and patients. However, the implementation of such systems should be carefully planned and organized in order to ensure quality, safety, and acceptance. The gathered view of different stakeholders is a great source of information to understand the barriers and enablers for implementation in a specific context. OBJECTIVE: This study aimed to understand the context and stakeholder perspectives related to the future implementation of a clinical decision support system for predicting readmissions of patients with heart failure. The study was part of a larger project involving model development, interface design, and implementation planning of the system. METHODS: Interviews were held with 12 stakeholders from the regional and municipal health care organizations to gather their views on the potential effects implementation of such a decision support system could have as well as barriers and enablers for implementation. Data were analyzed based on the categories defined in the nonadoption, abandonment, scale-up, spread, sustainability (NASSS) framework. RESULTS: Stakeholders had in general a positive attitude and curiosity toward AI-based decision support systems, and mentioned several barriers and enablers based on the experiences of previous implementations of information technology systems. Central aspects to consider for the proposed clinical decision support system were design aspects, access to information throughout the care process, and integration into the clinical workflow. The implementation of such a system could lead to a number of effects related to both clinical outcomes as well as resource allocation, which are all important to address in the planning of implementation. Stakeholders saw, however, value in several aspects of implementing such system, emphasizing the increased quality of life for those patients who can avoid being hospitalized. CONCLUSIONS: Several ideas were put forward on how the proposed AI system would potentially affect and provide value for patients, professionals, and the organization, and implementation aspects were important parts of that. A successful system can help clinicians to prioritize the need for different types of treatments but also be used for planning purposes within the hospital. However, the system needs not only technological and clinical precision but also a carefully planned implementation process. Such a process should take into consideration the aspects related to all the categories in the NASSS framework. This study further highlighted the importance to study stakeholder needs early in the process of development, design, and implementation of decision support systems, as the data revealed new information on the potential use of the system and the placement of the application in the care process.

Interdisciplinary Human-Centered AI for Hospital Readmission Prediction of Heart Failure Patients.

The evolution of clinical decision support (CDS) tools has been improved by usage of new technologies, yet there is an increased need to develop user-friendly, evidence-based, and expert-curated CDS solutions. In this paper, we show with a use-case how interdisciplinary expertise can be combined to develop CDS tool for hospital readmission prediction of heart failure patients. We also discuss how to make the tool integrated in clinical workflow by understanding end-user needs and have clinicians-in-the-loop during the different development stages.

Artificial Intelligence Applications in Health Care Practice: Scoping Review.

BACKGROUND: Artificial intelligence (AI) is often heralded as a potential disruptor that will transform the practice of medicine. The amount of data collected and available in health care, coupled with advances in computational power, has contributed to advances in AI and an exponential growth of publications. However, the development of AI applications does not guarantee their adoption into routine practice. There is a risk that despite the resources invested, benefits for patients, staff, and society will not be realized if AI implementation is not better understood. OBJECTIVE: The aim of this study was to explore how the implementation of AI in health care practice has been described and researched in the literature by answering 3 questions: What are the characteristics of research on implementation of AI in practice? What types and applications of AI systems are described? What characteristics of the implementation process for AI systems are discernible? METHODS: A scoping review was conducted of MEDLINE (PubMed), Scopus, Web of Science, CINAHL, and PsycINFO databases to identify empirical studies of AI implementation in health care since 2011, in addition to snowball sampling of selected reference lists. Using Rayyan software, we screened titles and abstracts and selected full-text articles. Data from the included articles were charted and summarized. RESULTS: Of the 9218 records retrieved, 45 (0.49%) articles were included. The articles cover diverse clinical settings and disciplines; most (32/45, 71%) were published recently, were from high-income countries (33/45, 73%), and were intended for care providers (25/45, 56%). AI systems are predominantly intended for clinical care, particularly clinical care pertaining to patient-provider encounters. More than half (24/45, 53%) possess no action autonomy but rather support human decision-making. The focus of most research was on establishing the effectiveness of interventions (16/45, 35%) or related to technical and computational aspects of AI systems (11/45, 24%). Focus on the specifics of implementation processes does not yet seem to be a priority in research, and the use of frameworks to guide implementation is rare. CONCLUSIONS: Our current empirical knowledge derives from implementations of AI systems with low action autonomy and approaches common to implementations of other types of information systems. To develop a specific and empirically based implementation framework, further research is needed on the more disruptive types of AI systems being implemented in routine care and on aspects unique to AI implementation in health care, such as building trust, addressing transparency issues, developing explainable and interpretable solutions, and addressing ethical concerns around privacy and data protection.

Realizing the potential of artificial intelligence in healthcare: Learning from intervention, innovation, implementation and improvement sciences.

Introduction: Artificial intelligence (AI) is widely seen as critical for tackling fundamental challenges faced by health systems. However, research is scant on the factors that influence the implementation and routine use of AI in healthcare, how AI may interact with the context in which it is implemented, and how it can contribute to wider health system goals. We propose that AI development can benefit from knowledge generated in four scientific fields: intervention, innovation, implementation and improvement sciences. Aim: The aim of this paper is to briefly describe the four fields and to identify potentially relevant knowledge from these fields that can be utilized for understanding and/or facilitating the use of AI in healthcare. The paper is based on the authors' experience and expertise in intervention, innovation, implementation, and improvement sciences, and a selective literature review. Utilizing knowledge from the four fields: The four fields have generated a wealth of often-overlapping knowledge, some of which we propose has considerable relevance for understanding and/or facilitating the use of AI in healthcare. Conclusion: Knowledge derived from intervention, innovation, implementation, and improvement sciences provides a head start for research on the use of AI in healthcare, yet the extent to which this knowledge can be repurposed in AI studies cannot be taken for granted. Thus, when taking advantage of insights in the four fields, it is important to also be explorative and use inductive research approaches to generate knowledge that can contribute toward realizing the potential of AI in healthcare.