Comparison of internal medicine applicant and resident characteristics with performance on ACGME milestones.

Internal medicine (IM) residency programs select applicants based on several metrics. Factors predicting success during residency are unclear across studies. To identify whether specific applicant or resident factors are associated with IM resident performance using ACGME milestones. We tested for associations between applicant factors available prior to the start of IM residency and resident factors measured during IM residency training, and resident performance on ACGME milestones across three consecutive years of IM training between 2015-2020. Univariable and multivariable linear regression modeling was used to test associations. Eighty-nine categorical IM residents that completed 3 consecutive years of training were included. Median age was 28 years (IQR 27-29) and 59.6% were male. Mean ACGME milestone scores increased with each post-graduate year (PGY) from 3.36 (SD 0.19) for PGY-1, to 3.80 (SD 0.15) for PGY-2, to 4.14 (SD 0.15) for PGY-3. Univariable modeling suggested referral to the clinical competency committee (CCC) for professionalism concerns was negatively associated with resident performance during each PGY. No applicant or resident factors included in the final multivariable regression models (age at starting residency, USMLE Step scores, interview score, rank list position, ITE scores) were associated with ACGME milestone scores for PGY-1 and PGY-2. Referral to the CCC for professionalism was negatively associated with resident performance during PGY-3. Residency selection factors did not predict resident milestone evaluation scores. Referral to the CCC was associated with significantly worse resident evaluation scores, suggesting professionalism may correlate with clinical performance.

A comparison of natural language processing to ICD-10 codes for identification and characterization of pulmonary embolism.

INTRODUCTION: The 10th revision of the International Classification of Diseases (ICD-10) codes is frequently used to identify pulmonary embolism (PE) events, although the validity of ICD-10 has been questioned. Natural language processing (NLP) is a novel tool that may be useful for pulmonary embolism identification. METHODS: We performed a retrospective comparative accuracy study of 1000 randomly selected healthcare encounters with a CT pulmonary angiogram ordered between January 1, 2019 and January 1, 2020 at a single academic medical center. Two independent observers reviewed each radiology report and abstracted key findings related to PE presence/absence, chronicity, and anatomic location. NLP interpretations of radiology reports and ICD-10 codes were queried electronically and compared to the reference standard, manual chart review. RESULTS: A total of 970 encounters were included for analysis. The prevalence of PE was 13% by manual review. For PE identification, sensitivity was similar between NLP (96.0%) and ICD-10 (92.9%; p = 0.405), and specificity was significantly higher with NLP (97.7%) compared to ICD-10 (91.0%; p < 0.001). NLP demonstrated higher sensitivity (70.0% vs 16.5%, p < 0.001) and specificity (99.9% vs 99.4%, p = 0.014) for saddle/main PE recognition, and significantly higher sensitivity (86.7% vs 8.3%, p < 0.001) and specificity (99.8% vs 96.5%, p < 0.001) for subsegmental PE compared to ICD-10. CONCLUSIONS: NLP is highly sensitive for PE identification and more specific than ICD-10 coding. NLP outperformed ICD-10 coding for recognition of subsegmental, saddle, and chronic PE. Our results suggest NLP is an efficient and more reliable method than ICD-10 for PE identification and characterization.

Comparison of Resident, Advanced Practice Clinician, and Hospitalist Teams in an Academic Medical Center: Association With Clinical Outcomes and Resource Utilization.

BACKGROUND: Academic medical centers have expanded their inpatient medicine services with advanced practice clinicians (APCs) or nonteaching hospitalists in response to patient volumes, residency work hour restrictions, and recently, COVID-19. Reports of clinical outcomes, cost, and resource utilization differ among inpatient team structures. OBJECTIVE: Directly compare outcomes among resident, APC, and solo hospitalist inpatient general medicine teams. DESIGN: Retrospective cohort study using multivariable analysis adjusted for time of admission, interhospital transfer, and comorbidities that compares clinical outcomes, cost, and resource utilization. SUBJECTS: Patients 18 years or older discharged from an inpatient medicine service between July 2015 and July 2018 (N = 12,716). MAIN MEASURES: Length of stay (LOS), 30-day readmission, inpatient mortality, normalized total direct cost, discharge time, and consultation utilization. KEY RESULTS: Resident teams admitted fewer patients at night (32.0%; P < .001) than did APC (49.5%) and hospitalist (48.6%) teams. APCs received nearly 4% more outside transfer patients (P = .015). Hospitalists discharged patients 26 minutes earlier than did residents (mean hours after midnight [95% CI], 14.58 [14.44-14.72] vs 15.02 [14.97-15.08]). Adjusted consult utilization was 15% higher for APCs (adjusted mean consults per admission [95% CI], 1.00 [0.96-1.03]) and 8% higher for residents (0.93 [0.90-0.95]) than it was for hospitalists (0.85 [0.80-0.90]). No differences in LOS, readmission, mortality, or cost were observed between the teams. CONCLUSION: We observed similar costs, LOS, 30-day readmission, and mortality among hospitalist, APC, and resident teams. Our results suggest clinical outcomes are not significantly affected by team structure. The addition of APC or hospitalist teams represent safe and effective alternatives to traditional inpatient resident teams.

Impact of Hospitalist Team Structure on Patient-Reported Satisfaction with Physician Performance.

BACKGROUND: Patient experience is valuable because it reflects how patients perceive the care they receive within the healthcare system and is associated with clinical outcomes. Also, as part of the Hospital Value-Based Purchasing (HVBP) program, the Center for Medicare and Medicaid Services (CMS) rewards hospitals with financial incentives for patient experience as measured by the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey. It is unclear how the addition of residents and advanced practice clinicians (APCs) to hospitalist-led inpatient teams affects patient satisfaction as measured by the HCAHPS and Press Ganey survey. OBJECTIVE: To compare patient satisfaction with hospitalists on resident, APC, and solo hospitalist teams measured by HCAHPS and Press Ganey physician performance domain survey results. DESIGN: Retrospective observational cohort study. PARTICIPANTS: All patients discharged from the Internal Medicine inpatient service between July 1, 2015, and July 1, 2018, who met HCAHPS survey eligibility criteria and completed a patient experience survey. MAIN MEASURES: HCAHPS and Press Ganey physician performance domain survey results. KEY RESULTS: No differences were observed in the selection of "top box" scores on the HCAHPS physician performance domain between resident, APC, and solo hospitalist teams. Adjusted Press Ganey physician performance domain survey results demonstrated significant differences between solo hospitalist and resident teams, with solo hospitalists having higher scores in three areas: time physician spent with you (4.58 vs. 4.38, p = 0.050); physician kept you informed (4.63 vs. 4.43, p = 0.047); and physician skill (4.80 vs. 4.63, p = 0.027). Solo hospitalists were perceived to have higher physician skill in comparison with hospitalist-APC teams (4.80 vs. 4.69, p = 0.042). CONCLUSION: While Press Ganey survey results suggest that patients have greater satisfaction with physicians on solo hospitalist teams, these differences were not observed on the HCAHPS physician performance survey domain, suggesting physician team structure does not impact HVBP incentive payments by CMS.