Making machine learning matter to clinicians: model actionability in medical decision-making.

Machine learning (ML) has the potential to transform patient care and outcomes. However, there are important differences between measuring the performance of ML models in silico and usefulness at the point of care. One lens to use to evaluate models during early development is actionability, which is currently undervalued. We propose a metric for actionability intended to be used before the evaluation of calibration and ultimately decision curve analysis and calculation of net benefit. Our metric should be viewed as part of an overarching effort to increase the number of pragmatic tools that identify a model's possible clinical impacts.

Development of Care Curves Following the Stage 1 Palliation: A Comparison of Intensive Care Among 5 Centers.

Background Comparison of care among centers is currently limited to major end points, such as mortality, length of stay, or complication rates. Creating "care curves" and comparing individual elements of care over time may highlight modifiable differences in intensive care among centers. Methods and Results We performed an observational retrospective study at 5 centers in the United States to describe key elements of postoperative care following the stage 1 palliation. A consecutive sample of 502 infants undergoing stage 1 palliation between January 2009 and December 2018 were included. All electronic health record entries relating to mandatory mechanical ventilator rate, opioid administration, and fluid intake/outputs between postoperative days (POD) 0 to 28 were extracted from each institution's data warehouse. During the study period, 502 patients underwent stage 1 palliation among the 5 centers. Patients were weaned to a median mandatory mechanical ventilator rate of 10 breaths/minute by POD 4 at Center 5 but not until POD 7 to 8 at Centers 1 and 2. Opioid administration peaked on POD 2 with extreme variance (median 6.9 versus 1.6 mg/kg per day at Center 3 versus Center 2). Daily fluid balance trends were variable: on POD 3 Center 1 had a median fluid balance of -51 mL/kg per day, ranging between -34 to 19 mL/kg per day among remaining centers. Intercenter differences persist after adjusting for patient and surgical characteristics (P<0.001 for each end point). Conclusions It is possible to detail and compare individual elements of care over time that represent modifiable differences among centers, which persist even after adjusting for patient factors. Care curves may be used to guide collaborative quality improvement initiatives.

Lessons Learned From the First Pilot Study of the Compensatory Reserve Index After Congenital Heart Surgery Requiring Cardiopulmonary Bypass.

BACKGROUND: Early warning systems that utilize dense physiologic data and machine learning may aid prediction of decompensation after congenital heart surgery (CHS). The Compensatory Reserve Index (CRI) analyzes changing features of the pulse waveform to predict hemodynamic decompensation in adults, but it has never been studied after CHS. This study sought to understand the feasibility, safety, and potential utility of CRI monitoring after CHS with cardiopulmonary bypass (CPB). METHODS: A single-center prospective pilot cohort of patients undergoing pulmonary valve replacement was studied. Compensatory Reserve Index was continuously measured from preoperative baseline through the first 24 postoperative hours. Average CRI values during selected procedural phases were compared between patients with an intensive care unit (ICU) length of stay (LOS) <3 days versus LOS ≥3 days. RESULTS: Twenty-three patients were enrolled. On average, 17,445 (±3,152) CRI data points were collected and 0.33% (±0.40) of data were missing per patient. There were no adverse events related to monitoring. Five (21.7%) patients had an ICU LOS ≥3 days. Compared to the ICU LOS <3 days group, the ICU LOS ≥3 days group had a greater decrease in CRI from baseline to immediately after CPB (-0.3 ± 0.1 vs -0.1 ± 0.2, P = .003) and were less likely to recover to baseline CRI during the monitoring period (20% vs 83%, P = .017). CONCLUSIONS: Compensatory Reserve Index monitoring after CHS with CPB seems feasible and safe. Early changes in CRI may precede meaningful clinical outcomes, but this requires further study.

The Compensatory Reserve Index Responds to Acute Hemodynamic Changes in Patients with Congenital Heart Disease: A Proof of Concept Study.

Patients with congenital heart disease (CHD) who undergo cardiac procedures may become hemodynamically unstable. Predictive algorithms that utilize dense physiologic data may be useful. The compensatory reserve index (CRI) trends beat-to-beat progression from normovolemia (CRI = 1) to decompensation (CRI = 0) in hemorrhagic shock by continuously analyzing unique sets of features in the changing pulse photoplethysmogram (PPG) waveform. We sought to understand if the CRI accurately reflects changing hemodynamics during and after a cardiac procedure for patients with CHD. A transcatheter pulmonary valve replacement (TcPVR) model was used because left ventricular stroke volume decreases upon sizing balloon occlusion of the right ventricular outflow tract (RVOT) and increases after successful valve placement. A single-center, prospective cohort study was performed. The CRI was continuously measured to determine the change in CRI before and after RVOT occlusion and successful TcPVR. Twenty-six subjects were enrolled with a median age of 19 (interquartile range (IQR) 13-29) years. The mean (± standard deviation) CRI decreased from 0.66 ± 0.15 1-min before balloon inflation to 0.53 ± 0.16 (p = 0.03) 1-min after balloon deflation. The mean CRI increased from a pre-valve mean CRI of 0.63 [95% confidence interval (CI) 0.56-0.70] to 0.77 (95% CI 0.71-0.83) after successful TcPVR. In this study, the CRI accurately reflected acute hemodynamic changes associated with TcPVR. Further research is justified to determine if the CRI can be useful as an early warning tool in patients with CHD at risk for decompensation during and after cardiac procedures.

Echocardiographic Measurements of Right Ventricular Mechanics in Infants with Bronchopulmonary Dysplasia at 36 Weeks Postmenstrual Age.

OBJECTIVE: To test the hypothesis that specific echocardiographic measurements of right ventricular (RV) mechanics at 36 weeks postmenstrual age (PMA) are associated with the severity of bronchopulmonary dysplasia (BPD). STUDY DESIGN: A subset of 93 preterm infants (born between 27 and 29 weeks of gestation) was selected retrospectively from a prospectively enrolled cohort. BPD was defined using the National Institutes of Health workshop definition, with modifications for oxygen reduction testing and altitude. The cohort was divided into no-BPD and BPD groups using previously published methodology for analyses. Echocardiographic measurements of RV function (ie, tricuspid annular plane systolic excursion, fractional area of change, systolic-to-diastolic ratio, tissue Doppler myocardial performance index, and RV strain), RV remodeling/morphology (end-systolic left ventricular eccentricity index), and RV afterload (pulmonary artery acceleration time measure) were evaluated at 36 weeks PMA. Multivariable logistic regression determined associations between RV measurements and BPD severity. RESULTS: Compared with the no-BPD cohort, the BPD group had lower birth weight z-scores (P = .04) and trended toward a male predominance (P = .08). After adjusting for birth weight z-score, gestational age, and sex, there were no between-group differences in echocardiographic measurements except for the eccentricity index (scaled OR [0.1-unit increase], 1.49; 95% CI, 1.13-2.12; P = .01). CONCLUSIONS: Among conventional and emerging echocardiographic measurements of RV mechanics, eccentricity index was the sole variable independently associated with BPD severity in this study. The eccentricity index may be a useful echocardiographic measurement for characterizing RV mechanics in patients with BPD at 36 weeks PMA.

Toward standardization of care: The feeding readiness assessment after congenital cardiac surgery.

BACKGROUND: Feeding practices after neonatal and congenital heart surgery are complicated and variable, which may be associated with prolonged hospitalization length of stay (LOS). Systematic assessment of feeding skills after cardiac surgery may earlier identify those likely to have protracted feeding difficulties, which may promote standardization of care. METHODS: Neonates and infants ≤3 months old admitted for their first cardiac surgery were retrospectively identified during a 1-year period at a single center. A systematic feeding readiness assessment (FRA) was utilized to score infant feeding skills. FRA scores were assigned immediately prior to surgery and 1, 2, and 3 weeks after surgery. FRA scores were analyzed individually and in combination as predictors of gastrostomy tube (GT) placement prior to hospital discharge by logistic regression. RESULTS: Eighty-six patients met inclusion criteria and 69 patients had complete data to be included in the final model. The mean age of admit was five days and 51% were male. Forty-six percent had single ventricle physiology. Twenty-nine (42%) underwent GT placement. The model containing both immediate presurgical and 1-week postoperative FRA scores was of highest utility in predicting discharge with GT (intercept odds = 10.9, P = .0002; sensitivity 69%, specificity 93%, AUC 0.913). The false positive rate was 7.5%. CONCLUSIONS: In this analysis, systematic and standardized measurements of feeding readiness employed immediately before and one week after congenital cardiac surgery predicted need for GT placement prior to hospital discharge. The FRA score may be used to risk stratify patients based on likelihood of prolonged feeding difficulties, which may further improve standardization of care.

Noncompliance to a Postoperative Algorithm Using Feeding Readiness Assessments Prolonged Length of Stay at a Pediatric Heart Institute.

INTRODUCTION: Variable compliance to postoperative feeding algorithms after pediatric cardiac surgery may be associated with suboptimal growth, decreased parental satisfaction, and prolonged hospital length of stay (LOS). Our heart center performed an audit of compliance to a previously introduced postoperative feeding algorithm to guide quality improvement efforts. We hypothesized that algorithm noncompliance would be associated with increased LOS. METHODS: We retrospectively identified children ≤ 3 months admitted for their first cardiac surgery between January 1, 2015 and December 31, 2016. The algorithm uses objective oral feeding readiness assessments (FRA). At the end of a predefined evaluation period, a "sentinel" FRA score is assigned. The sentinel FRA and FRA trend guide decisions to pursue gastrostomy tube (GT) or oral-only feeds. Among those who reached the sentinel FRA, we defined compliance as ≤ 3 days before pursuing GT or oral-only feeds once indicated by the algorithm. RESULTS: Sixty-nine patients were included. Forty-nine complied with the algorithm (71%), and 45 received GT (65.2%). Noncompliers had significantly longer LOS (34 versus 25 days; P = 0.01). Among GT recipients, noncompliers waited 6 additional days for a GT compared with compliers (P ≤ 0.001). Subjective decisions to extend oral feeding trials or await results of a swallow study were associated with algorithm noncompliance. CONCLUSIONS: This audit of compliance to a feeding algorithm after pediatric cardiac surgery highlighted variability of practice, including relying on subjective appraisals of feeding skills over objective FRAs. This variability was associated with increased LOS and can be hypothesis-generating for future quality improvement efforts.

Overwhelmed and uninspired by lack of coordinated care: a call to action for new physicians.

In this commentary, the author argues that medical students and residents often find themselves overwhelmed with the realities of facilitating coordinated care. He suggests that the difficulty in obtaining outside records for newly transferred patients leads to both unnecessary duplication of procedures and learned helplessness for new physicians. He provides examples of care coordination challenges by describing his five-hour attempt to obtain the medical records and brain biopsy slides for a transfer patient. He points out that practical solutions to these challenges have already been proposed but that their implementation remains a challenge given the many invested stakeholders. He calls on medical students and residents to join together with patients, community advocates, departmental leaders, and health system administrators to demand changes to the care coordination system that will ultimately provide patients with more efficient and appropriate medical care.

Slow and steady: readiness, pretreatment weekly strengthening activity, and pediatric weight management program completion.

BACKGROUND: Pediatric weight management programs have substantial attrition rates, which have led to recommendations to assess readiness prior to enrollment. Both pretreatment readiness scales and behaviors, such as exercise, have been theorized to predict subsequent program completion. The purpose of this study was to explore the role of self-reported pretreatment exercise in adolescents on completion of a pediatric weight management program and to explore the predictive ability of standard readiness scales. METHODS: A total of 146 obese (BMI≥95(th) percentile) pediatric (ages 11-18) participants joined a 6-month multidisciplinary weight management program between March, 2007, and July, 2010. Completers were compared retrospectively to noncompleters on demographic, readiness, and pretreatment exercise practices from clinic-developed intake questionnaires using univariate analyses. Regression analyses specified the degree to which these variables predicted program completion. RESULTS: The 6-month completion rate was 53%. There was no relationship between self-reported readiness and program completion. Self-reported pretreatment weekly strengthening activity (SA) was significantly associated with program completion, compared to those who performed SA either never [univariate odds ratio (OR) 3.18, 95% confidence interval (CI) 1.51-6.68, p=0.002; multivariate OR 2.43, 95% CI 1.06-5.58, p=0.036] or daily (univariate OR 4.90, 95% CI 1.74-13.77, p=0.002; multivariate OR 4.69, 95% CI 1.45-15.14, p=0.010). No relationship was found between other forms of exercise and program completion. CONCLUSIONS: Self-reported pretreatment weekly SA, but not standard readiness scales, predicted pediatric weight management program completion.