Framework for Classifying Explainable Artificial Intelligence (XAI) Algorithms in Clinical Medicine.

Artificial intelligence (AI) applied to medicine offers immense promise, in addition to safety and regulatory concerns. Traditional AI produces a core algorithm result, typically without a measure of statistical confidence or an explanation of its biological-theoretical basis. Efforts are underway to develop explainable AI (XAI) algorithms that not only produce a result but also an explanation to support that result. Here we present a framework for classifying XAI algorithms applied to clinical medicine: An algorithm's clinical scope is defined by whether the core algorithm output leads to observations (eg, tests, imaging, clinical evaluation), interventions (eg, procedures, medications), diagnoses, and prognostication. Explanations are classified by whether they provide empiric statistical information, association with a historical population or populations, or association with an established disease mechanism or mechanisms. XAI implementations can be classified based on whether algorithm training and validation took into account the actions of health care providers in response to the insights and explanations provided or whether training was performed using only the core algorithm output as the end point. Finally, communication modalities used to convey an XAI explanation can be used to classify algorithms and may affect clinical outcomes. This framework can be used when designing, evaluating, and comparing XAI algorithms applied to medicine.

Artificial intelligence modelling to assess the risk of cardiovascular disease in oncology patients.

Aims: There are no comprehensive machine learning (ML) tools used by oncologists to assist with risk identification and referrals to cardio-oncology. This study applies ML algorithms to identify oncology patients at risk for cardiovascular disease for referrals to cardio-oncology and to generate risk scores to support quality of care. Methods and results: De-identified patient data were obtained from Vanderbilt University Medical Center. Patients with breast, kidney, and B-cell lymphoma cancers were targeted. Additionally, the study included patients who received immunotherapy drugs for treatment of melanoma, lung cancer, or kidney cancer. Random forest (RF) and artificial neural network (ANN) ML models were applied to analyse each cohort: A total of 20 023 records were analysed (breast cancer, 6299; B-cell lymphoma, 9227; kidney cancer, 2047; and immunotherapy for three covered cancers, 2450). Data were divided randomly into training (80%) and test (20%) data sets. Random forest and ANN performed over 90% for accuracy and area under the curve (AUC). All ANN models performed better than RF models and produced accurate referrals. Conclusion: Predictive models are ready for translation into oncology practice to identify and care for patients who are at risk of cardiovascular disease. The models are being integrated with electronic health record application as a report of patients who should be referred to cardio-oncology for monitoring and/or tailored treatments. Models operationally support cardio-oncology practice. Limited validation identified 86% of the lymphoma and 58% of the kidney cancer patients with major risk for cardiotoxicity who were not referred to cardio-oncology.

Grounded in reality: artificial intelligence in medical education.

Background: In a recent survey, medical students expressed eagerness to acquire competencies in the use of artificial intelligence (AI) in medicine. It is time that undergraduate medical education takes the lead in helping students develop these competencies. We propose a solution that integrates competency-driven AI instruction in medical school curriculum. Methods: We applied constructivist and backwards design principles to design online learning assignments simulating the real-world work done in the healthcare industry. Our innovative approach assumed no technical background for students, yet addressed the need for training clinicians to be ready to practice in the new digital patient care environment. This modular 4-week AI course was implemented in 2019, integrating AI with evidence-based medicine, pathology, pharmacology, tele-monitoring, quality improvement, value-based care, and patient safety. Results: This educational innovation was tested in 2 cohorts of fourth year medical students who demonstrated an improvement in knowledge with an average quiz score of 97% and in skills with an average application assignment score of 89%. Weekly reflections revealed how students learned to transition from theory to practice of AI and how these concepts might apply to their upcoming residency training programs and future medical practice. Conclusions: We present an innovative product that achieves the objective of competency-based education of students regarding the role of AI in medicine. This course can be integrated in the preclinical years with a focus on foundational knowledge, vocabulary, and concepts, and in clinical years with a focus on application of core knowledge to real-world scenarios.

Innovation of health data science curricula.

Objective: There is a growing need for innovation to prepare a well-trained health informatics workforce with data science and digital technology skills. To meet the workforce demands and prepare students for a career in health informatics, a Health Data Science (HDS) concentration was added to the Master's in Health Informatics (MSHI) program at the University of Illinois at Chicago. Methods: Four levels of learning were incorporated into the curriculum to prepare students for highly complex jobs in health informatics. Leader interviews, advisory board meetings, and mixed faculty expertise were utilized as inputs to survey and analyze the skills employers seek in the job market. An innovative rapid infusion approach was used to design assessments across the levels of learning that simulate real-world scenarios where these competencies are used. Results: Course evaluation surveys revealed strong satisfaction with the quality of the course and agreed that the course was intellectually challenging and stimulating. Students reported the 3 most beneficial aspects were: the live lectures, hands-on data research and manipulation, and simulated real-world situations. Conclusions: This article discusses using a rapid infusion approach to developing active learning assignments designed to build competencies employers are seeking. These competencies also develop creative, divergent thinking with flexible, student-defined solutions. Survey data validates the approach to active learning put into context and made relevant to the learner. The benefit of the concentration is to provide students with the preparation for a successful entry into the Health Informatics field, one of the fastest-growing careers in healthcare.

From Testing to Decision-Making: A Data-Driven Analytics COVID-19 Response.

In March 2020, NorthShore University Health System laboratories mobilized to develop and validate polymerase chain reaction based testing for detection of SARS-CoV-2. Using laboratory data, NorthShore University Health System created the Data Coronavirus Analytics Research Team to track activities affected by SARS-CoV-2 across the organization. Operational leaders used data insights and predictions from Data Coronavirus Analytics Research Team to redeploy critical care resources across the hospital system, and real-time data were used daily to make adjustments to staffing and supply decisions. Geographical data were used to triage patients to other hospitals in our system when COVID-19 detected pavilions were at capacity. Additionally, one of the consequences of COVID-19 was the inability for patients to receive elective care leading to extended periods of pain and uncertainty about a disease or treatment. After shutting down elective surgeries beginning in March of 2020, NorthShore University Health System set a recovery goal to achieve 80% of our historical volumes by October 1, 2020. Using the Data Coronavirus Analytics Research Team, our operational and clinical teams were able to achieve 89% of our historical volumes a month ahead of schedule, allowing rapid recovery of surgical volume and financial stability. The Data Coronavirus Analytics Research Team also was used to demonstrate that the accelerated recovery period had no negative impact with regard to iatrogenic COVID-19 infection and did not result in increased deep vein thrombosis, pulmonary embolisms, or cerebrovascular accident. These achievements demonstrate how a coordinated and transparent data-driven effort that was built upon a robust laboratory testing capability was essential to the operational response and recovery from the COVID-19 crisis.

ICD-10 procedure codes produce transition challenges.

The transition of procedure coding from ICD-9-CM-Vol-3 to ICD-10-PCS has generated problems for the medical community at large resulting from the lack of clarity required to integrate two non-congruent coding systems. We hypothesized that quantifying these issues with network topology analyses offers a better understanding of the issues, and therefore we developed solutions (online tools) to empower hospital administrators and researchers to address these challenges. Five topologies were identified: "identity"(I), "class-to-subclass"(C2S), "subclass-toclass"(S2C), "convoluted(C)", and "no mapping"(NM). The procedure codes in the 2010 Illinois Medicaid dataset (3,290 patients, 116 institutions) were categorized as C=55%, C2S=40%, I=3%, NM=2%, and S2C=1%. Majority of the problematic and ambiguous mappings (convoluted) pertained to operations in ophthalmology cardiology, urology, gyneco-obstetrics, and dermatology. Finally, the algorithms were expanded into a user-friendly tool to identify problematic topologies and specify lists of procedural codes utilized by medical professionals and researchers for mitigating error-prone translations, simplifying research, and improving quality.http://www.lussiergroup.org/transition-to-ICD10PCS.

Effectiveness of Evidence-based Pneumonia CPOE Order Sets Measured by Health Outcomes.

OBJECTIVE: Evidence-based sets of medical orders for the treatment of patients with common conditions have the potential to induce greater efficiency and convenience across the system, along with more consistent health outcomes. Despite ongoing utilization of order sets, quantitative evidence of their effectiveness is lacking. In this study, conducted at Advocate Health Care in Illinois, we quantitatively analyzed the benefits of community acquired pneumonia order sets as measured by mortality, readmission, and length of stay (LOS) outcomes. METHODS: In this study, we examined five years (2007-2011) of computerized physician order entry (CPOE) data from two city and two suburban community care hospitals. Mortality and readmissions benefits were analyzed by comparing "order set" and "no order set" groups of adult patients using logistic regression, Pearson's chi-squared, and Fisher's exact methods. LOS was calculated by applying one-way ANOVA and the Mann-Whitney U test, supplemented by analysis of comorbidity via the Charlson Comorbidity Index. RESULTS: The results indicate that patient treatment orders placed via electronic sets were effective in reducing mortality [OR=1.787; 95% CF 1.170-2.730; P=.061], readmissions [OR=1.362; 95% CF 1.015-1.827; P=.039], and LOS [F (1,5087)=6.885, P=.009, 4.79 days (no order set group) vs. 4.32 days (order set group)]. CONCLUSION: Evidence-based ordering practices have the potential to improve pneumonia outcomes through reduction of mortality, hospital readmissions, and cost of care. However, the practice must be part of a larger strategic effort to reduce variability in patient care processes. Further experimental and/or observational studies are required to reduce the barriers to retrospective patient care analyses.

Patient Outcomes as Transformative Mechanisms to Bring Health Information Technology Industry and Research Informatics Closer Together.

Despite the fast pace of recent innovation within the health information technology and research informatics domains, there remains a large gap between research and academia, while interest in translating research innovations into implementations in the patient care settings is lacking. This is due to absence of common outcomes and performance measurement targets, with health information technology industry employing financial and operational measures and academia focusing on patient outcome concerns. The paper introduces methodology for and roadmap to introduction of common objectives as a way to encourage better collaboration between industry and academia using patient outcomes as a composite measure of demonstrated success from health information systems investments. Along the way, the concept of economics of health informatics, or "infonomics," is introduced to define a new way of mapping future technology investments in accordance with projected clinical impact.

The complexity and challenges of the International Classification of Diseases, Ninth Revision, Clinical Modification to International Classification of Diseases, 10th Revision, Clinical Modification transition in EDs.

Beginning October 2015, the Center for Medicare and Medicaid Services will require medical providers to use the vastly expanded International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) system. Despite wide availability of information and mapping tools for the next generation of the ICD classification system, some of the challenges associated with transition from ICD-9-CM to ICD-10-CM are not well understood. To quantify the challenges faced by emergency physicians, we analyzed a subset of a 2010 Illinois Medicaid database of emergency department ICD-9-CM codes, seeking to determine the accuracy of existing mapping tools in order to better prepare emergency physicians for the change to the expanded ICD-10-CM system. We found that 27% of 1830 codes represented convoluted multidirectional mappings. We then analyzed the convoluted transitions and found that 8% of total visit encounters (23% of the convoluted transitions) were clinically incorrect. The ambiguity and inaccuracy of these mappings may impact the workflow associated with the translation process and affect the potential mapping between ICD codes and Current Procedural Codes, which determine physician reimbursement.

Effectiveness of Evidence-Based Congestive Heart Failure (CHF) CPOE Order Sets Measured by Health Outcomes.

OBJECTIVE: Evidence-based order sets for treatment of patients with common conditions promise ordering efficiency and more consistent health outcomes. Despite ongoing utilization of order sets, quantitative evidence of their effectiveness is lacking. This study quantitatively analyzed benefits of CHF order sets as measured by mortality, readmission, and length of stay (LOS) outcomes. METHODS: Mortality and readmissions were analyzed by comparing "order set" and "free text" groups of adult patients using logistic regression, Pearson chi-squared, and Fisher's exact methods. LOS was calculated by applying One-Way ANOVA and Mann-Whitney tests, supplemented by comorbidity analysis via Charlson Comorbidity Index. RESULTS: CHF orders placed via sets were effective in reducing mortality [OR=1.818;95% CF 1.039-3.181;p=0.034] and LOS [F(1,10938)=8.352,p=0.013,4.75 days ("free text" group) vs. 5.46 days ("order set" group)], while readmission outcome was not significant [OR=0.913;95% CF 0.734-1.137;p=0.417]. CONCLUSION: Evidence-based medication ordering practices to treat CHF have potential to reduce mortality and LOS, without effect on readmissions.

Building effective workforce management practices through shared governance and technology systems integration.

In integrated delivery networks (IDNs) with complex management structures, shared governance in nursing is a proven model for health care delivery. After Advocate Health Care, the largest IDN in Illinois, implemented shared governance in its nursing, clinical, and non-clinical departments and restructured the organization's technology use, it benefited greatly from a new, shared decision-making process. After listening to business consultants, clinical professionals, and information technology experts, hospitals should take the blended, or comprehensive, approach to new projects. They can succeed by promoting communication supported by an integrated computer platform that helps nursing and business executives reach a consensus. Traditional modes of operation, in which individual administrative, clinical, and technology departments separately introduce innovation, do not deliver an advantage. However, models that incorporate open communication, integration, and knowledge sharing will help large IDNs and other complex health care organizations make the best possible use of their resources and investments.