MOUNTAINEER: Topology-Driven Visual Analytics for Comparing Local Explanations.

With the increasing use of black-box Machine Learning (ML) techniques in critical applications, there is a growing demand for methods that can provide transparency and accountability for model predictions. As a result, a large number of local explainability methods for black-box models have been developed and popularized. However, machine learning explanations are still hard to evaluate and compare due to the high dimensionality, heterogeneous representations, varying scales, and stochastic nature of some of these methods. Topological Data Analysis (TDA) can be an effective method in this domain since it can be used to transform attributions into uniform graph representations, providing a common ground for comparison across different explanation methods. We present a novel topology-driven visual analytics tool, Mountaineer, that allows ML practitioners to interactively analyze and compare these representations by linking the topological graphs back to the original data distribution, model predictions, and feature attributions. Mountaineer facilitates rapid and iterative exploration of ML explanations, enabling experts to gain deeper insights into the explanation techniques, understand the underlying data distributions, and thus reach well-founded conclusions about model behavior. Furthermore, we demonstrate the utility of Mountaineer through two case studies using real-world data. In the first, we show how Mountaineer enabled us to compare black-box ML explanations and discern regions of and causes of disagreements between different explanations. In the second, we demonstrate how the tool can be used to compare and understand ML models themselves. Finally, we conducted interviews with three industry experts to help us evaluate our work.

Clinical outcomes of patients with acute myeloid leukemia and cardiovascular disease.

Incidence of both acute myeloid leukemia (AML) and cardiovascular disease (CVD) increases with age. We evaluated whether pre-existing CVD impacts clinical outcomes in AML. We retrospectively evaluated 291 consecutive adult AML patients treated at our institution, 2014-2020. Pretreatment comorbidities were identified by chart review. Outcomes included complete remission (CR) and CR with incomplete count recovery (CRi) rates, disease-free survival (DFS), overall survival (OS) and incidence of cardiovascular adverse events. CVD was present in 34% of patients at AML diagnosis. CVD patients had worse performance status (p=0.03) and more commonly had secondary AML (p=0.03) and received hypomethylating (HMA) agent-based therapy (72% vs 38%, p< 0.001). CVD (0.45 vs 0.71, p<0.001) and diabetes mellitus (HR= 0.24, 95% CI: 0.08 - 0.8, p= 0.01) were associated with lower probability of achieving CR/CRi. Accounting for age, performance status (PS), complex karyotype, secondary disease and treatment, CVD patients had shorter OS (HR=1.5, 95% CI: 1.1-2.2, p=0.002), with 1- and 3-year OS 44% vs 67% and 25% vs 40%, respectively, but there was no difference in cumulative incidence of relapse between patients with vs without CVD. Thus, CVD is an independent risk factor for lower response rate and shorter survival in AML patients.

Clozapine and neutrophil response in patients of African descent: A six-month, multinational, prospective, open-label clinical trial.

INTRODUCTION: Clozapine is the most effective antipsychotic for treatment-resistant schizophrenia, but it is markedly underutilized, particularly in the US Black population, partly because of concern over clozapine-associated low absolute neutrophil count (ANC). People of African descent have a lower normative ANC range than the White population, which is associated with a specific "ACKR1-null" ("Duffy null") CC genotype (SNP rs2814778) on the ACKR1 gene, termed benign ethnic neutropenia (BEN). The range of ANC variability and safety of clozapine have not been established in people with BEN or examined prospectively in people of African descent. METHODS: We completed a multisite, 6-month, prospective, open-label clinical trial of clozapine treatment in people of African descent with schizophrenia spectrum disorders for whom clozapine was clinically indicated, with or without the ACKR1-null genotype. We examined clozapine safety and weekly ANC during clozapine treatment and evaluated ANC variability by ACKR1-null genotype, sex, study site, and clozapine dosing using repeated measures analysis of covariance. Genotype was assayed using TaqMan® technology. RESULTS: We enrolled 274 participants, of whom 227 (82.8 %) completed 6 months of clozapine treatment. There was one case of severe neutropenia (<500 cells/mm3) (0.36 %) over 1467.6 person-months of clozapine exposure. This participant recovered without sequelae after discontinuation of clozapine. Of the 249 participants with known genotypes, 199 (79.9 %) had the ACKR1-null genotype. Neutropenia (<1500 cells/mm3) occurred significantly more often in the ACKR1-null group (33 % [65/199]) than in those with the T allele (6 % (3/50); p < 0.001). Fourteen (5 %) patients discontinued due to adverse events. Rates of infection and fever were low and sialorrhea was the commonest side effect (N = 187, 68 %). CONCLUSION: To our knowledge, this is the largest prospective clozapine trial in people of African descent. Severe neutropenia was rare, despite the high prevalence (80 %) of the ACKR1-null genotype. Our findings suggest that clozapine can be used safely in Black patients including those with BEN.

Hypertension, smoking, and preexistence of multiple cardiac risk factors correlate with carfilzomib-induced cardiovascular adverse events in a racially diverse population.

Background: Use of the proteasome inhibitor carfilzomib has become a standard of care in patients with relapsed/refractory multiple myeloma. An association between carfilzomib and cardiovascular adverse events has been well documented, but this had not been investigated in a racially diverse population. Black patients in particular are underrepresented in the reported outcomes of treatment with carfilzomib. Objective: The purpose of this study was to identify risk factors for carfilzomib-associated cardiovascular events in a diverse, single-center population. Methods: We conducted a retrospective review of 161 patients with multiple myeloma treated with carfilzomib between 2011 and 2020 at the University of Maryland Medical Center. Over half (86) were Black patients, with the remainder (75) being White patients. We did a multivariate analysis to determine risk factors for developing cardiovascular events during treatment with carfilzomib. Results: There was no statistically significant association with cardiotoxicity and race, gender, or age at first dose of carfilzomib. In multivariable analysis, patients with history of hypertension had a higher risk of cardiotoxicity [adjusted odds ratio (OR): 2.5; 95% CI: 1.1-5.9; P = 0.03] as did those with a history of smoking [OR: 2.8; 95% CI: 1.3-6.4; P = 0.01]. Conclusions: Here we report the largest cohort of Black patients treated with carfilzomib as yet reported. The results of this single center retrospective study show history of hypertension and smoking are associated with carfilzomib associated cardiotoxicity in a diverse patient population. There is a need for well-designed prospective studies enrolling a diverse population to investigate potential interventions to prevent carfilzomib-associated cardiotoxicity.

Calibrate: Interactive Analysis of Probabilistic Model Output.

Analyzing classification model performance is a crucial task for machine learning practitioners. While practitioners often use count-based metrics derived from confusion matrices, like accuracy, many applications, such as weather prediction, sports betting, or patient risk prediction, rely on a classifier's predicted probabilities rather than predicted labels. In these instances, practitioners are concerned with producing a calibrated model, that is, one which outputs probabilities that reflect those of the true distribution. Model calibration is often analyzed visually, through static reliability diagrams, however, the traditional calibration visualization may suffer from a variety of drawbacks due to the strong aggregations it necessitates. Furthermore, count-based approaches are unable to sufficiently analyze model calibration. We present Calibrate, an interactive reliability diagram that addresses the aforementioned issues. Calibrate constructs a reliability diagram that is resistant to drawbacks in traditional approaches, and allows for interactive subgroup analysis and instance-level inspection. We demonstrate the utility of Calibrate through use cases on both real-world and synthetic data. We further validate Calibrate by presenting the results of a think-aloud experiment with data scientists who routinely analyze model calibration.

Visual Exploration of Machine Learning Model Behavior with Hierarchical Surrogate Rule Sets.

One of the potential solutions for model interpretation is to train a surrogate model: a more transparent model that approximates the behavior of the model to be explained. Typically, classification rules or decision trees are used due to their logic-based expressions. However, decision trees can grow too deep, and rule sets can become too large to approximate a complex model. Unlike paths on a decision tree that must share ancestor nodes (conditions), rules are more flexible. However, the unstructured visual representation of rules makes it hard to make inferences across rules. In this paper, we focus on tabular data and present novel algorithmic and interactive solutions to address these issues. First, we present Hierarchical Surrogate Rules (HSR), an algorithm that generates hierarchical rules based on user-defined parameters. We also contribute SuRE, a visual analytics (VA) system that integrates HSR and an interactive surrogate rule visualization, the Feature-Aligned Tree, which depicts rules as trees while aligning features for easier comparison. We evaluate the algorithm in terms of parameter sensitivity, time performance, and comparison with surrogate decision trees and find that it scales reasonably well and overcomes the shortcomings of surrogate decision trees. We evaluate the visualization and the system through a usability study and an observational study with domain experts. Our investigation shows that the participants can use feature-aligned trees to perform non-trivial tasks with very high accuracy. We also discuss many interesting findings, including a rule analysis task characterization, that can be used for visualization design and future research.

SUBPLEX: A Visual Analytics Approach to Understand Local Model Explanations at the Subpopulation Level.

Understanding the interpretation of machine learning (ML) models has been of paramount importance when making decisions with societal impacts, such as transport control, financial activities, and medical diagnosis. While local explanation techniques are popular methods to interpret ML models on a single instance, they do not scale to the understanding of a model's behavior on the whole dataset. In this article, we outline the challenges and needs of visually analyzing local explanations and propose SUBPLEX, a visual analytics approach to help users understand local explanations with subpopulation visual analysis. SUBPLEX provides steerable clustering and projection visualization techniques that allow users to derive interpretable subpopulations of local explanations with users' expertise. We evaluate our approach through two use cases and experts' feedback.

Prognostic Value of Red Blood Cell Distribution Width in Transcatheter Aortic Valve Replacement Patients.

OBJECTIVE: Elevated red blood cell distribution width (RDW) level has been shown to be associated with poor outcomes in patients with cardiovascular disease. Limited data are available regarding the prognostic value of RDW in transcatheter aortic valve replacement (TAVR) patients. Therefore, we aimed to investigate the impact of RDW variation on outcomes of TAVR patients. METHODS: From March 20, 2012, to February 20, 2020, the pre-TAVR RDW levels of 1,163 consecutive TAVR patients were examined. Receiver operating curves were set to define the most accurate cut-point, which was subsequently validated in our validation set. Associations of RDW levels with early and long-term outcomes were investigated. RESULTS: A total of 988 patients were eligible for the analysis. Patients with 30-day, 1-year, and 7-year mortality had significantly higher pre-TAVR RDW levels (15.8% [12.9-19.1] vs 14.7% [11.6-26.3], P = 0.01; 16% [12.3-26.3] vs 14.7% [11.6-24.3], P < 0.001; 15.6% [12.3-26.3] vs 14.6% [11.6-24.3], P < 0.001, respectively). A RDW of 14.5% was found as the most sensitive and specific cut-point for mortality at 1 and 7 years (HR = 2.6, 95% CI: 1.6-4.2, P < 0.001; HR = 1.8, 95% CI: 1.3-2.4, P < 0.001), with mortality of 22% versus 10% at 1 year (P < 0.001) and 37% versus 27% at 7 years (P < 0.001) in patients with RDW ≥14.5% versus those with RDW <14.5%. CONCLUSIONS: RDW is an important prognostic factor in TAVR patients. A RDW level higher than 14.5% is significantly associated with post-TAVR early and late mortality. RDW levels should be incorporated into current risk assessment models as an additional variable to predict post-TAVR outcomes.

Acute Valvular Heart Disease.

Valvular heart disease (VHD) is a common clinical entity. Recognition of decompensated VHD is crucial to instituting appropriate workup and management. Initial evaluation focuses on hemodynamics, peripheral perfusion, volume overload, and active myocardial ischemia. Initial therapy is targeted at improving hemodynamics, fluid status, and decreasing myocardial ischemia before intervention. Echocardiography can rapidly identify VHD etiology and severity along with physical examination findings. Owing to improved survival with cardiac surgery over the past several decades, prosthetic valve dysfunction should be recognized and initial treatment understood. Mechanical circulatory support is increasingly part of clinical practice in stabilizing patients with decompensated VHD.

Left Ventricular Pseudoaneurysm Following Inferior Myocardial Infarction: A Case for Conservative Management.

Left ventricular pseudoaneurysm is a rare complication of myocardial infarction that carries a high mortality rate. Although conventional wisdom suggests prompt surgical repair in order to mitigate risk of expansion and rupture, there are some data to support non-operative management in asymptomatic individuals with likely chronic pseudoaneurysms, particularly when surgical candidacy is poor. We present a case of a medically managed left ventricular pseudoaneurysm subsequent to inferior ST-segment elevation myocardial infarction with 6-month follow-up data.

4-Methyl-7-thioumbelliferyl-beta-D-cellobioside: a fluorescent, nonhydrolyzable substrate analogue for cellulases.

The kinetics of cellulose binding and hydrolysis by cellulases is not well understood except at steady-state conditions. For use in studies of cellulase pre-steady-state and steady-state kinetics, we have prepared 4-methyl-7-thioumbelliferyl-beta-D-cellobioside (MUS-CB), a ground-state nonhydrolyzable analogue of the fluorescent cellulase substrate 4-methylumbelliferyl-beta-D-cellobioside (MU-CB). MUS-CB is not hydrolyzed by the catalytic domain of cellulase E1 from Acidothermus cellulolyticus under conditions where this enzyme rapidly degrades MU-CB. Thermodynamic parameters describing the steady-state binding of MUS-CB to Thermobifida fusca cellulase Cel6A are similar to those for MU-CB, indicating that MUS-CB can be used in place of MU-CB to study binding events in the Cel6A active-site cleft. In the pre-steady-state, MUS-CB binds to Cel6A by a simple, one-step bimolecular association reaction. It is anticipated that similar thio-containing 4-methylumbelliferyl compounds will have applications in studies of other enzyme systems.