Real-Time Student Feedback on the Surgical Learning Environment: Use of a Mobile Application.

OBJECTIVE: Improvements to the medical student surgical learning environment are limited by lack of granular data and recall bias on end-of-clerkship evaluations. The purpose of this study was to identify specific areas for intervention using a novel real-time mobile application. DESIGN: An application was designed to obtain real-time feedback from medical students regarding the learning environment on their surgical clerkship. Thematic analysis of student experiences was performed at the conclusion of 4 consecutive 12-week rotation blocks. SETTING: Brigham and Women's Hospital/Harvard Medical School, Boston, Massachusetts. RESULTS: Fifty-four medical students at a single institution were asked to participate during their primary clerkship experience. Students submitted 365 responses over 48 weeks. Multiple themes emerged which were dichotomized into positive and negative emotions centered on specific student priorities. Approximately half of responses were associated with positive emotions (52.9%) and half with negative emotions (47.1%). Student priorities included the desire to feel included in the surgical team (resulting in feeling engaged/ignored), to have a positive relationship with members of the team (perceiving kind/rude interactions), to witness compassionate patient care (observing empathy/disrespect for patients), to have a well-planned surgical rotation (experiencing organization/disorganization within teams), and to feel that student well-being is prioritized (reporting opportunities/disregard for student wellness). CONCLUSION: A novel, user-friendly mobile application identified several areas to improve the experience and engagement of students on their surgery clerkship. Allowing clerkship directors and other educational leaders to collect longitudinal data in real time may allow for more targeted, timely improvements to the medical student surgical learning environment.

Comparison of the PROMIS Preference Score (PROPr) and EQ-5D-5L Index Value in General Population Samples in the United Kingdom, France, and Germany.

OBJECTIVES: The Patient-Reported Outcome Measurement Information System (PROMIS) Preference score (PROPr) can be used to assess health state utility (HSU) and estimate quality-adjusted life-years in cost-effectiveness analyses. It is based on item response theory and promises to overcome limitations of existing HSU scores such as ceiling effects. The PROPr contains 7 PROMIS domains: cognitive abilities, depression, fatigue, pain, physical function, sleep disturbance, and ability to participate in social roles and activities. We aimed to compare the PROPr with the 5-level EQ-5D (EQ-5D-5L) in terms of psychometric properties using data from 3 countries. METHODS: We collected PROMIS-29 profile and EQ-5D-5L data from 3 general population samples (United Kingdom = 1509, France = 1501, Germany = 1502). Given that cognition is not assessed by the PROMIS-29, it was predicted by the recommended linear regression model. We compared the convergent validity, known-groups construct validity, and ceiling and floor effects of the PROPr and EQ-5D-5L. RESULTS: The mean PROPr (0.48, 0.53, 0.48; P<.01) and EQ-5D-5L scores (0.82, 0.85, 0.83; P<.01) showed significant differences of similar magnitudes (d = 0.34; d = 0.32; d = 0.35; P<.01) across all samples. The differences were invariant to sex, income, occupation, education, and most conditions but not for age. The Pearson correlation coefficients between both scores were r = 0.74, r = 0.69, and r = 0.72. PROPr's ceiling and floor effects both were minor to moderate. The EQ-5D-5L's ceiling (floor) effects were major (negligible). CONCLUSIONS: Both the EQ-5D-5L and the PROPr assessed by the PROMIS-29 show high validity. The PROPr yields considerably lower HSU values than the EQ-5D-5L. Consequences for quality-adjusted life-year measurements should be investigated in future research.

Feasibility and performance of smartphone-based daily micro-surveys among patients recovering from cancer surgery.

AIMS: Daily micro-surveys, or the high-frequency administration of patient-reported outcome measures (PROMs), may provide real-time, unbiased assessments of health-related quality of life (HRQoL). We evaluated the feasibility and accuracy of daily micro-surveys using a smartphone platform among patients recovering from cancer surgery. METHODS: In a prospective study (2017-2019), patients undergoing cancer surgery downloaded a smartphone application that administered daily micro-surveys comprising five randomly selected items from the Short Form-36 (SF-36). Micro-surveys were administered without replacement until the entire SF-36 was administered weekly. The full-length SF-36 was also administered preoperatively and 4, 12, and 24 weeks postoperatively. We assessed response and completion rates between the micro-surveys and full-length SF-36, as well as agreement of responses using Bland-Altman (B&A) analyses. RESULTS: Ninety-five patients downloaded the application and were followed for a mean of 131 days [SD ± 85]. Response rates for the full-length SF-36 and micro-surveys was 76% [95%CI 69, 83], and 34% [95%CI 26, 39]. Despite lower response rates, more SF-36 surveys were collected using the daily micro-surveys compared to the intermittent full-length SF-36 (9.9 [95%CI 8.4, 12.6] vs. 3.0 [95%CI 2.8, 3.3], respectively). B&A analyses demonstrated lack of agreement between micro-surveys and SF-36. However, agreement improved with higher micro-survey completion rate. Eighty-five percent of participants reported that daily micro-surveys were not burdensome. CONCLUSION: This study suggests that collection of daily micro-surveys among patients recovering from cancer surgery is feasible using smartphones in the early postoperative period. Future implementation of daily micro-surveys may more granularly describe momentary HRQoL changes through a greater volume of self-reported survey data.