Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data.

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Our results illustrate how the widely-used Accuracy (Acc) metric, which measures the overall proportion of successful predictions, yields misleadingly high performances, as class imbalance increases. Because Acc weights the per-class ratios of correct predictions proportionally to class size, it largely disregards the performance on the minority class. A binary classification model that learns to systematically vote for the majority class will yield an artificially high decoding accuracy that directly reflects the imbalance between the two classes, rather than any genuine generalizable ability to discriminate between them. We show that other evaluation metrics such as the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC), and the less common Balanced Accuracy (BAcc) metric - defined as the arithmetic mean between sensitivity and specificity, provide more reliable performance evaluations for imbalanced data. Our findings also highlight the robustness of Random Forest (RF), and the benefits of using stratified cross-validation and hyperprameter optimization to tackle data imbalance. Critically, for neuroscience ML applications that seek to minimize overall classification error, we recommend the routine use of BAcc, which in the specific case of balanced data is equivalent to using standard Acc, and readily extends to multi-class settings. Importantly, we present a list of recommendations for dealing with imbalanced data, as well as open-source code to allow the neuroscience community to replicate and extend our observations and explore alternative approaches to coping with imbalanced data.

Race and Insurance Impact on Hospital Admission From the Emergency Department for Patients Diagnosed With Heart Failure.

OBJECTIVE: The purpose of this study was to analyze if patient race and the presence of insurance predict the odds of admission from the emergency department (ED) for patients diagnosed with congestive heart failure (CHF). BACKGROUND: Excessive hospital readmissions for patients with CHF are considered a quality-of-care issue. Previous studies have not considered race and insurance in conjunction with quality measures in predicting hospital admission from the ED for these patients. METHODS: A secondary data analysis was conducted from cross-sectional archival data from the 2015 National Hospital Ambulatory Medical Care Survey using cross-tabulations with χ followed by multiple logistic regression analysis. RESULTS: Race and the presence of insurance were not significant in predicting the odds of admission from the ED for patients with CHF. CONCLUSIONS: Being seen in the ED within the last 72 hours and seen by provider types consulting physician and nurse practitioner were significant (P ≤ .05) in predicting the odds of admission related to a diagnosis of CHF.

Elevated hemostasis markers after pneumonia increases one-year risk of all-cause and cardiovascular deaths.

BACKGROUND: Acceleration of chronic diseases, particularly cardiovascular disease, may increase long-term mortality after community-acquired pneumonia (CAP), but underlying mechanisms are unknown. Persistence of the prothrombotic state that occurs during an acute infection may increase risk of subsequent atherothrombosis in patients with pre-existing cardiovascular disease and increase subsequent risk of death. We hypothesized that circulating hemostasis markers activated during CAP persist at hospital discharge, when patients appear to have recovered clinically, and are associated with higher mortality, particularly due to cardiovascular causes. METHODS: In a cohort of survivors of CAP hospitalization from 28 US sites, we measured D-Dimer, thrombin-antithrombin complexes [TAT], Factor IX, antithrombin, and plasminogen activator inhibitor-1 at hospital discharge, and determined 1-year all-cause and cardiovascular mortality. RESULTS: Of 893 subjects, most did not have severe pneumonia (70.6% never developed severe sepsis) and only 13.4% required intensive care unit admission. At discharge, 88.4% of subjects had normal vital signs and appeared to have clinically recovered. D-dimer and TAT levels were elevated at discharge in 78.8% and 30.1% of all subjects, and in 51.3% and 25.3% of those without severe sepsis. Higher D-dimer and TAT levels were associated with higher risk of all-cause mortality (range of hazard ratios were 1.66-1.17, p = 0.0001 and 1.46-1.04, p = 0.001 after adjusting for demographics and comorbid illnesses) and cardiovascular mortality (p = 0.009 and 0.003 in competing risk analyses). CONCLUSIONS: Elevations of TAT and D-dimer levels are common at hospital discharge in patients who appeared to have recovered clinically from pneumonia and are associated with higher risk of subsequent deaths, particularly due to cardiovascular disease.

Rate of palivizumab administration in accordance with current recommendations among hospitalized children.

INTRODUCTION: The American Academy of Pediatrics recommends palivizumab prophylaxis for children born premature or with chronic lung disease to reduce the severity of respiratory syncytial virus (RSV) infection. The purpose of the current study is to examine palivizumab administration among children hospitalized with RSV infection. METHOD: This is a retrospective medical record review at a tertiary care children's hospital. The study included children <2 years of age hospitalized between October 1, 2000, and April 1, 2001, with the diagnosis of RSV infection. RESULTS: The records of 264 children meeting inclusion criteria were reviewed. Forty children qualified for administration of palivizumab using American Academy of Pediatrics recommendations. Of these, 14 (35%) received palivizumab prior to admission. Palivizumab administration rate was not affected by age, race, or insurance coverage. DISCUSSION: Exact barriers to the administration of palivizumab remain unclear. The identification of high-risk children, prevention of RSV by use of palivizumab, and collaboration between hospital and community health care providers will help increase the use of palivizumab and decrease the incidence of RSV.

Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review.

CONTEXT: Intensive care unit (ICU) physician staffing varies widely, and its association with patient outcomes remains unclear. OBJECTIVE: To evaluate the association between ICU physician staffing and patient outcomes. DATA SOURCES: We searched MEDLINE (January 1, 1965, through September 30, 2001) for the following medical subject heading (MeSH) terms: intensive care units, ICU, health resources/utilization, hospitalization, medical staff, hospital organization and administration, personnel staffing and scheduling, length of stay, and LOS. We also used the following text words: staffing, intensivist, critical, care, and specialist. To identify observational studies, we added the MeSH terms case-control study and retrospective study. Although we searched for non-English-language citations, we reviewed only English-language articles. We also searched EMBASE, HealthStar (Health Services, Technology, Administration, and Research), and HSRPROJ (Health Services Research Projects in Progress) via Internet Grateful Med and The Cochrane Library and hand searched abstract proceedings from intensive care national scientific meetings (January 1, 1994, through December 31, 2001). STUDY SELECTION: We selected randomized and observational controlled trials of critically ill adults or children. Studies examined ICU attending physician staffing strategies and the outcomes of hospital and ICU mortality and length of stay (LOS). Studies were selected and critiqued by 2 reviewers. We reviewed 2590 abstracts and identified 26 relevant observational studies (of which 1 included 2 comparisons), resulting in 27 comparisons of alternative staffing strategies. Twenty studies focused on a single ICU. DATA SYNTHESIS: We grouped ICU physician staffing into low-intensity (no intensivist or elective intensivist consultation) or high-intensity (mandatory intensivist consultation or closed ICU [all care directed by intensivist]) groups. High-intensity staffing was associated with lower hospital mortality in 16 of 17 studies (94%) and with a pooled estimate of the relative risk for hospital mortality of 0.71 (95% confidence interval [CI], 0.62-0.82). High-intensity staffing was associated with a lower ICU mortality in 14 of 15 studies (93%) and with a pooled estimate of the relative risk for ICU mortality of 0.61 (95% CI, 0.50-0.75). High-intensity staffing reduced hospital LOS in 10 of 13 studies and reduced ICU LOS in 14 of 18 studies without case-mix adjustment. High-intensity staffing was associated with reduced hospital LOS in 2 of 4 studies and ICU LOS in both studies that adjusted for case mix. No study found increased LOS with high-intensity staffing after case-mix adjustment. CONCLUSIONS: High-intensity vs low-intensity ICU physician staffing is associated with reduced hospital and ICU mortality and hospital and ICU LOS.