Reproducibility in machine learning for health research: Still a ways to go.

Machine learning for health must be reproducible to ensure reliable clinical use. We evaluated 511 scientific papers across several machine learning subfields and found that machine learning for health compared poorly to other areas regarding reproducibility metrics, such as dataset and code accessibility. We propose recommendations to address this problem.

Continuous Digital Assessment for Weight Loss Surgery Patients.

We conducted a survey about recent surgical procedures on a large connected population and requested each individual's permission to access data from commercial wearable devices they may have been wearing around the time of the procedure. For subcohorts of 66-118 patients who reported having a weight loss procedure and who had dense Fitbit data around their procedure date, we examined several daily measures of behavior and physiology in the 12 weeks leading up to and the 12 weeks following their procedures. We found that the weeks following weight loss operations were associated with fewer daily total steps, smaller proportions of the day spent walking, lower resting and 95th percentile heart rates, more total sleep time, and greater sleep efficiency. We demonstrate that consumer-grade activity trackers can capture behavioral and physiological changes resulting from weight loss surgery and these devices have the potential to be used to develop measures of patients' postoperative recovery that are convenient, sensitive, scalable, individualized, and continuous.

Predicting Subjective Recovery from Lower Limb Surgery Using Consumer Wearables.

INTRODUCTION: A major challenge in the monitoring of rehabilitation is the lack of long-term individual baseline data which would enable accurate and objective assessment of functional recovery. Consumer-grade wearable devices enable the tracking of individual everyday functioning prior to illness or other medical events which necessitate the monitoring of recovery trajectories. METHODS: For 1,324 individuals who underwent surgery on a lower limb, we collected their Fitbit device data of steps, heart rate, and sleep from 26 weeks before to 26 weeks after the self-reported surgery date. We identified subgroups of individuals who self-reported surgeries for bone fracture repair (n = 355), tendon or ligament repair/reconstruction (n = 773), and knee or hip joint replacement (n = 196). We used linear mixed models to estimate the average effect of time relative to surgery on daily activity measurements while adjusting for gender, age, and the participant-specific activity baseline. We used a sub-cohort of 127 individuals with dense wearable data who underwent tendon/ligament surgery and employed XGBoost to predict the self-reported recovery time. RESULTS: The 1,324 study individuals were all US residents, predominantly female (84%), white or Caucasian (85%), and young to middle-aged (mean age 36.2 years). We showed that 12 weeks pre- and 26 weeks post-surgery trajectories of daily behavioral measurements (steps sum, heart rate, sleep efficiency score) can capture activity changes relative to an individual's baseline. We demonstrated that the trajectories differ across surgery types, recapitulate the documented effect of age on functional recovery, and highlight differences in relative activity change across self-reported recovery time groups. Finally, using a sub-cohort of 127 individuals, we showed that long-term recovery can be accurately predicted, on an individual level, only 1 month after surgery (AUROC 0.734, AUPRC 0.8). Furthermore, we showed that predictions are most accurate when long-term, individual baseline data are available. DISCUSSION: Leveraging long-term, passively collected wearable data promises to enable relative assessment of individual recovery and is a first step towards data-driven intervention for individuals.