Changes in Stress and Workplace Shortages Reported by U.S. Critical Care Physicians Treating Coronavirus Disease 2019 Patients.

OBJECTIVES: Eleven months into the coronavirus disease 2019 pandemic, the country faces accelerating rates of infections, hospitalizations, and deaths. Little is known about the experiences of critical care physicians caring for the sickest coronavirus disease 2019 patients. Our goal is to understand how high stress levels and shortages faced by these physicians during Spring 2020 have evolved. DESIGN: We surveyed (October 23, 2020 to November 16, 2020) U.S. critical care physicians treating coronavirus disease 2019 patients who participated in a National survey earlier in the pandemic (April 23, 2020 to May 3, 2020) regarding their stress and shortages they faced. SETTING: ICU. PATIENTS: Coronavirus disease 2019 patients. INTERVENTION: Irrelevant. MEASUREMENT: Physician emotional distress/physical exhaustion: low (not at all/not much), moderate, or high (a lot/extreme). Shortage indicators: insufficient ICU-trained staff and shortages in medication, equipment, or personal protective equipment requiring protocol changes. MAIN RESULTS: Of 2,375 U.S. critical care attending physicians who responded to the initial survey, we received responses from 1,356 (57.1% response rate), 97% of whom (1,278) recently treated coronavirus disease 2019 patients. Two thirds of physicians (67.6% [864]) reported moderate or high levels of emotional distress in the Spring versus 50.7% (763) in the Fall. Reports of staffing shortages persisted with 46.5% of Fall respondents (594) reporting a staff shortage versus 48.3% (617) in the Spring. Meaningful shortages of medication and equipment reported in the Spring were largely alleviated. Although personal protective equipment shortages declined by half, they remained substantial. CONCLUSIONS: Stress, staffing, and, to a lesser degree, personal protective equipment shortages faced by U.S. critical care physicians remain high. Stress levels were higher among women. Considering the persistence of these findings, rising levels of infection nationally raise concerns about the capacity of the U.S. critical care system to meet ongoing and future demands.

Opportunities for Maintenance of Certification to Better Reflect Scope of Practice Among Medical Oncologists.

PURPOSE: Critics argue that the American Board of Internal Medicine's medical oncology Maintenance of Certification examination requires medical oncologists with a narrow scope of practice to spend time studying material that is no longer relevant to their practice. However, no data are available describing the scope of practice for medical oncologists. METHODS: Using Medicare claims, we examined the scope of practice for 9,985 medical oncologists who saw 8.6 million oncology conditions in 2016, each of which was assigned to 1 of 23 different condition groups. Scope of practice was then measured as the percentage of oncology conditions within each of the 23 groups. We grouped physicians with similar scopes of practice by applying K-means clustering to the percentage of conditions seen. The scope of practice for each physician cluster was determined from the cancers that encompassed the majority of average oncology conditions seen among physicians composing the cluster. RESULTS: We found 20 distinct scope-of-practice clusters. The largest (n = 6,479 [65.5%]) had a general oncology scope of practice. The remaining physicians focused on a narrow scope of cancers, including 22.6% focused on ≥ 1 solid tumors and 11.9% focused on hematologic malignancies. The largest focused cluster accounted for 7.7% of physicians focused on breast cancer. CONCLUSION: A single American Board of Internal Medicine Maintenance of Certification assessment in medical oncology is most appropriate for approximately 65% of certified medical oncologists' practices. However, the addition of assessments focused on breast cancer and hematologic malignancies could increase this figure to upwards of 85% of certified medical oncologists.

Assessing the Quality of Osteoporosis Care in Practice.

BACKGROUND: Patients with osteoporosis can sustain fractures following falls or other minimal trauma. This risk of fracture can be reduced through appropriate diagnostic testing, pharmacologic therapy, and other readily measured standards of care. OBJECTIVES: Our aim was to develop a credible clinical performance assessment to measure physicians' quality of osteoporosis care, and determine reasonable performance standards for both competent and excellent care. DESIGN: This was a retrospective cohort study. PARTICIPANTS: Three hundred and eighty one general internists and subspecialists with time-limited board certification were included in the study. MAIN MEASURES: Performance rates on eight evidence-based measures were obtained from the American Board of Internal Medicine (ABIM) Osteoporosis Practice Improvement Module® (PIM), a web-based tool that uses medical chart reviews to help physicians assess and improve care. We applied a patented methodology, using an adaptation of the Angoff standard-setting method and the Dunn-Rankin method, with an expert panel skilled in osteoporosis care to form a composite and establish standards for both competent and excellent care. Physician and practice characteristics, including a practice infrastructure score based on the Physician Practice Connections Readiness Survey (PPC-RS), were used to examine the validity of the inferences made from the composite scores. KEY RESULTS: The mean composite score was 67.54 out of 100 maximum points with a reliability of 0.92. The standard for competent care was 46.87, and for excellent care it was 83.58. Both standards had high classification accuracies (0.95). Sixteen percent of physicians performed below the competent care standard, while 22 % met the excellent care standard. Specialists scored higher than generalists, and better practice infrastructure was associated with higher composite scores, providing some validity evidence. CONCLUSIONS: We developed a rigorous methodology for assessing physicians' osteoporosis care. Clinical performance feedback relative to absolute standards of care provides physicians with a meaningful approach to self-evaluation to improve patient care.

Association between imposition of a Maintenance of Certification requirement and ambulatory care-sensitive hospitalizations and health care costs.

IMPORTANCE: In 1990, the American Board of Internal Medicine (ABIM) ended lifelong certification by initiating a 10-year Maintenance of Certification (MOC) program that first took effect in 2000. Despite the importance of this change, there has been limited research examining associations between the MOC requirement and patient outcomes. OBJECTIVE: To measure associations between the original ABIM MOC requirement and outcomes of care. DESIGN, SETTING, AND PARTICIPANTS: Quasi-experimental comparison between outcomes for Medicare beneficiaries treated in 2001 by 2 groups of ABIM-certified internal medicine physicians (general internists). One group (n = 956), initially certified in 1991, was required to fulfill the MOC program in 2001 (MOC-required) and treated 84 215 beneficiaries in the sample; the other group (n = 974), initially certified in 1989, was grandfathered out of the MOC requirement (MOC-grandfathered) and treated 69 830 similar beneficiaries in the sample. We compared differences in outcomes for the beneficiary cohort treated by the MOC-required general internists before (1999-2000) and after (2002-2005) they were required to complete MOC, using the beneficiary cohort treated by the MOC-grandfathered general internists as the control. MAIN OUTCOMES AND MEASURES: Quality measures were ambulatory care-sensitive hospitalizations (ACSHs), measured using prevention quality indicators. Ambulatory care-sensitive hospitalizations are hospitalizations triggered by conditions thought to be potentially preventable through better access to and quality of outpatient care. Other outcomes included health care cost measures (adjusted to 2013 dollars). RESULTS: Annual incidence of ACSHs (per 1000 beneficiaries) increased from the pre-MOC period (37.9 for MOC-required beneficiaries vs 37.0 for MOC-grandfathered beneficiaries) to the post-MOC period (61.8 for MOC-required beneficiaries vs 61.4 for MOC-grandfathered beneficiaries) for both cohorts, as did annual per-beneficiary health care costs (pre-MOC period, $5157 for MOC-required beneficiaries vs $5133 for MOC-grandfathered beneficiaries; post-MOC period, $7633 for MOC-required beneficiaries vs $7793 for MOC-grandfathered beneficiaries). The MOC requirement was not statistically associated with cohort differences in the growth of the annual ACSH rate (per 1000 beneficiaries, 0.1 [95% CI, -1.7 to 1.9]; P = .92), but was associated with a cohort difference in the annual, per-beneficiary cost growth of -$167 (95% CI, -$270.5 to -$63.5; P = .002; 2.5% of overall mean cost). CONCLUSION AND RELEVANCE: Imposition of the MOC requirement was not associated with a difference in the increase in ACSHs but was associated with a small reduction in the growth differences of costs for a cohort of Medicare beneficiaries.

Development of an instrument to evaluate residents' confidence in quality improvement.

BACKGROUND: Practice-based learning and improvement is a core competency that all medical residents must demonstrate. Because confidence is important in translating competence into action, effective quality improvement (QI) curricula should evaluate trainees' knowledge and confidence to perform QI. Past efforts to assess educational outcomes in QI have not adequately evaluated trainees' confidence from a multidimensional perspective. METHODS: Participants--732 internal medicine and family medicine residents from 42 training programs in the United States--completed the 31-item Quality Improvement Confidence Instrument (QICI), which was developed to measure confidence in six QI skill domains based on the Institute for Healthcare Improvement model ofQI. Confirmatory factor analysis was performed to support construct validity. Multivariate analysis of covariance was used to examine associations between residents' QI experience and other characteristics with confidence scores. RESULTS: Confirmatory factor analysis supported the QICI's multidimensional structure. Individual items yielded adequate variability, and reliability estimates for all six domains were high (> 0.86). On average, residents rated their confidence lowest for skills pertaining to choosing a target for improvement (specifically, using methods to evaluate interventions and to identify sources of process errors) and for testing a change in practice using specific tools for data collection and analysis. After controlling for program year and other characteristics, residents with previous QI experience reported significantly greater QI confidence. CONCLUSION: The QICI offers a psychometrically rigorous approach to evaluating residents' confidence levels. It can be used to gauge the appropriateness of a trainee's confidence against actual QI performance.

A new tool to give hospitalists feedback to improve interprofessional teamwork and advance patient care.

Teamwork is a vital skill for health care professionals, but the fragmented systems within which they work frequently do not recognize or support good teamwork. The American Board of Internal Medicine has developed and is testing the Teamwork Effectiveness Assessment Module (TEAM), a tool for physicians to evaluate how they perform as part of an interprofessional patient care team. The assessment provides hospitalist physicians with feedback data drawn from their own work of caring for patients, in a way that is intended to support immediate, concrete change efforts to improve the quality of patient care. Our approach demonstrates the value of looking at teamwork in the real world of health care-that is, as it occurs in the actual contexts in which providers work together to care for patients. The assessment of individual physicians' teamwork competencies may play a role in the larger effort to bring disparate health professions together in a system that supports and rewards a team approach in hope of improving patient care.

Comparative trial of a web-based tool to improve the quality of care provided to older adults in residency clinics: modest success and a tough road ahead.

PURPOSE: To determine whether residency programs can use a multicomponent, Web-based quality improvement tool to improve the care of older adults. METHOD: The authors conducted an exploratory, cluster-randomized, comparative before-after trial of the Care of the Vulnerable Elderly Practice Improvement Module in the ambulatory clinics of 46 internal medicine and family medicine residency programs, 2006-2008. The main outcomes were the deltas between pre- and post-performance on the Assessing Care of the Vulnerable Elderly (ACOVE) quality measures. RESULTS: Of the 46 programs initially selected for the study, 37 (80%) provided both baseline and follow-up data. Performance on all 10 ACOVE measures was poor at baseline (range 8.6%-33.6%). Intervention clinics most frequently chose for improvement fall-risk screening and documentation of end-of-life preferences. The change in the percentage of patients screened for fall risk for the intervention clinics that targeted this measure was significantly greater than the change observed by the control clinics (+23.3% versus +9.7%, P = .003, odds ratio [OR] = 2.0; 95% confidence interval [CI]: 1.25-3.75), as was the difference observed for documentation of preference for life-sustaining care (+16.4% versus +2.8%, P = .002, OR = 6.3; 95% CI: 2.0-19.6) and surrogate decision maker (+14.3% versus +2.8%, P = .003, OR = 6.8; 95% CI: 1.9-24.4). CONCLUSIONS: A multicomponent, Web-based, quality improvement tool can help residency programs improve care for older adults, but much work remains for improving the state of care for this population in training settings.

Variability in obtaining institutional review board approval for quality improvement activities in residency programs.

INTRODUCTION: Quality improvement (QI) activities are an important part of residency training. National studies are needed to inform best practices in QI training and experience for residents. The impact of the Institutional Review Board (IRB) process on such studies is not well described. METHODS: This observational study looked at time, length, comfort level, and overall quality of experience for 42 residency training programs in obtaining approval or exemption for a nationally based educational QI study. RESULTS: For the 42 programs in the study, the time period to IRB approval/exemption was highly variable, ranging from less than 1 week to 56.5 weeks; mean and median time was approximately 18 weeks (SD, 10.8). Greater reported comfort with the IRB process was associated with less time to obtain approval (r  =  -.50; P < .01; 95% CI, -0.70 to -0.23). A more positive overall quality of experience with the IRB process was also associated with less time to obtain IRB approval (r  =  -.60; P < .01; 95% CI, -0.74 to -0.36). DISCUSSION: The IRB process for residency programs initiating QI studies shows considerable variance that is not explained by attributes of the projects. New strategies are needed to assist and expedite IRB processes for QI research in educational settings and reduce interinstitutional variability and increase comfort level among educators with the IRB process.

Listening to older adults: elderly patients' experience of care in residency and practicing physician outpatient clinics.

The population of people aged 65 and older is rapidly growing. Research has demonstrated significant quality gaps in the clinical care of older patients in the United States, especially in training programs. Little is known about how older patients' experience with care delivered in residency clinics compares with that delivered by practicing physicians. Using patient surveys from the American Board of Internal Medicine Care of the Vulnerable Elderly Practice Improvement Module, the quality of care provided to adults aged 65 and older by 52 internal medicine and family medicine residency clinics and by a group of 144 practicing physicians was studied. The residency clinics received 2,213 patient surveys, and the practicing physicians received 4,204. Controlling for age and overall health status, patients from the residency clinic sample were less likely to report receiving guidance and interventions for important aspects of care for older adults than patients from the practicing physician sample. The largest difference was observed in providing ways to help patients prevent falls or treat problems with balance or walking (42.1% vs 61.8%, P<.001). Patients from the residency clinic sample were less likely to rate their overall care as high (77.5% vs 88.8%, P<.001). Patient surveys reveal important deficiencies in processes of care that are more pronounced for patients cared for in residency clinics. Quality of patient experience and communication are vital aspects of overall quality of care, especially for older adults. Physician education at all levels, faculty development, and practice system redesign are needed to ensure that the care needs of older adults are met.

Setting a fair performance standard for physicians' quality of patient care.

BACKGROUND: Assessing physicians' clinical performance using statistically sound, evidence-based measures is challenging. Little research has focused on methodological approaches to setting performance standards to which physicians are being held accountable. OBJECTIVE: Determine if a rigorous approach for setting an objective, credible standard of minimally-acceptable performance could be used for practicing physicians caring for diabetic patients. DESIGN: Retrospective cohort study. PARTICIPANTS: Nine hundred and fifty-seven physicians from the United States with time-limited certification in internal medicine or a subspecialty. MAIN MEASURES: The ABIM Diabetes Practice Improvement Module was used to collect data on ten clinical and two patient experience measures. A panel of eight internists/subspecialists representing essential perspectives of clinical practice applied an adaptation of the Angoff method to judge how physicians who provide minimally-acceptable care would perform on individual measures to establish performance thresholds. Panelists then rated each measure's relative importance and the Dunn-Rankin method was applied to establish scoring weights for the composite measure. Physician characteristics were used to support the standard-setting outcome. KEY RESULTS: Physicians abstracted 20,131 patient charts and 18,974 patient surveys were completed. The panel established reasonable performance thresholds and importance weights, yielding a standard of 48.51 (out of 100 possible points) on the composite measure with high classification accuracy (0.98). The 38 (4%) outlier physicians who did not meet the standard had lower ratings of overall clinical competence and professional behavior/attitude from former residency program directors (p = 0.01 and p = 0.006, respectively), lower Internal Medicine certification and maintenance of certification examination scores (p = 0.005 and p < 0.001, respectively), and primarily worked as solo practitioners (p = 0.02). CONCLUSIONS: The standard-setting method yielded a credible, defensible performance standard for diabetes care based on informed judgment that resulted in a reasonable, reproducible outcome. Our method represents one approach to identifying outlier physicians for intervention to protect patients.

Measuring physicians' performance in clinical practice: reliability, classification accuracy, and validity.

Much research has been devoted to addressing challenges in achieving reliable assessments of physicians' clinical performance but less work has focused on whether valid and accurate classification decisions are feasible. This study used 957 physicians certified in internal medicine (IM) or a subspecialty, who completed the American Board of Internal Medicine (ABIM) Diabetes Practice Improvement Module (PIM). Ten clinical and two patient-experience measures were aggregated into a composite measure. The composite measure score was highly reliable (r = .91) and classification accuracy was high across the entire score scale (>0.90), which indicated that it is possible to differentiate high-performing and low-performing physicians. Physicians certified in endocrinology and those who scored higher on their IM certification examination had higher composite scores, providing some validity evidence. In summary, it is feasible to create a psychometrically robust composite measure of physicians' clinical performance, specifically for the quality of care they provide to patients with diabetes.

Clinic systems and the quality of care for older adults in residency clinics and in physician practices.

PURPOSE: The U.S. health care system is not prepared to meet the needs of the increasing population of older adults. Few physicians become geriatricians, but most will care for older adults. The authors assessed the quality of care for older adults in residency clinics and physician practices. METHOD: Using the American Board of Internal Medicine's Care of the Vulnerable Elderly practice improvement module, researchers studied the quality of care provided to older adults in 52 internal and family medicine residency clinic sites and by a motivated group of 144 practicing physicians from 2006 to 2008. They also studied the characteristics of the practice systems in the clinics and offices and the relationship between specific elements of practice systems and the quality of care. RESULTS: Patients seen by residents were younger, had fewer chronic conditions, and were less likely to receive recommended care. Residency clinic systems were less likely to have elements designed to support care for older adults. Even when present, there was little correlation with care provided. Practicing physicians were more likely to provide recommended processes of care, and system elements in their practices were more likely to function well and correlate with delivery of key processes of care, but much room for improvement remains. CONCLUSIONS: Practice system elements designed to support care for older adults perform differently in residency clinics than in practicing physicians' offices. Significant gaps in the quality of care for older adults exist and are much more pronounced in the residency clinic setting.

Toward better care coordination through improved communication with referring physicians.

BACKGROUND: Effective care coordination requires good physician-to-physician communication. The authors evaluated a new tool called the Communication with Referring Physicians Practice Improvement Module (CRP-PIM), which assesses and encourages improved communication among physician consultants and referring physicians. METHOD: Eight-hundred three consultants (internists and subspecialists) completed a practice system survey and were rated by 12,212 referring physicians on 13 communication processes using a six-point scale. Consultants received an interactive performance report and selected targets for improvement. Data were analyzed using descriptive statistics, correlations, t tests, and factor analysis. RESULTS: Mean overall rating was high, at 5.53 (SD 0.23, range 2.46-5.95); consultants still identified areas for improvement. The generalizability coefficient for overall ratings was 0.78. Factor analysis supported two categories of ratings associated with consultants' gender, subspecialty, residency performance ratings, and specific practice system features. CONCLUSIONS: The CRP-PIM provides a psychometrically viable measure and encourages consultants to improve communication with referring physicians.

Variation in internal medicine residency clinic practices: assessing practice environments and quality of care.

BACKGROUND: Few studies have systematically and rigorously examined the quality of care provided in educational practice sites. OBJECTIVE: The objectives of this study were to (1) describe the patient population cared for by trainees in internal medicine residency clinics; (2) assess the quality of preventive cardiology care provided to these patients; (3) characterize the practice-based systems that currently exist in internal medicine residency clinics; and (4) examine the relationships between quality, practice-based systems, and features of the program: size, type of program, and presence of an electronic medical record. DESIGN: This is a cross-sectional observational study. SETTING: This study was conducted in 15 Internal Medicine residency programs (23 sites) throughout the USA. PARTICIPANTS: The participants included site champions at residency programs and 709 residents. MEASUREMENTS: Abstracted charts provided data about patient demographics, coronary heart disease risk factors, processes of care, and clinical outcomes. Patients completed surveys regarding satisfaction. Site teams completed a practice systems survey. RESULTS: Chart abstraction of 4,783 patients showed substantial variability across sites. On average, patients had between 3 and 4 of the 9 potential risk factors for coronary heart disease, and approximately 21% had at least 1 important barrier of care. Patients received an average of 57% (range, 30-77%) of the appropriate interventions. Reported satisfaction with care was high. Sites with an electronic medical record showed better overall information management (81% vs 27%) and better modes of communication (79% vs 43%). CONCLUSIONS: This study has provided insight into the current state of practice in residency sites including aspects of the practice environment and quality of preventive cardiology care delivered. Substantial heterogeneity among the training sites exists. Continuous measurement of the quality of care provided and a better understanding of the training environment in which this care is delivered are important goals for delivering high quality patient care.

Self-assessment of practice performance: development of the ABIM Practice Improvement Module (PIM).

BACKGROUND: Quality measurement and improvement in practice are requirements for Maintenance of Certification by the American Board of Medical Specialties boards and a component of many pay for performance programs. OBJECTIVE: To describe the development of the American Board of Internal Medicine (ABIM) Practice Improvement Module (PIM) and the average performance of ABIM diplomates who have completed the Preventive Cardiology PIM. DESIGN: Observational study of self-administered practice quality improvement. SETTING: Office practices through the United States. PARTICIPANTS: A total of 179 cardiologists and general internists completing requirements for ABIM Maintenance of Certification from 2004 through 2005. MEASUREMENTS: Physicians self-audited at least 25 charts to obtain performance measures, patient demographics, and coronary heart disease risk factors. At least 25 patients completed surveys regarding their experience of care in the physician's practice. Physicians completed a self-assessment survey detailing the presence of various practice systems. RESULTS: The mean rate for systolic blood pressure control was 48%, for diastolic blood pressure 84%, and for low-density lipoprotein (LDL) cholesterol at goal 65%. Of patients 61% rated the quality of care as excellent and 58% rated the practices excellent at encouraging questions and answering them clearly. More than 85% of patients reported "no problem" obtaining a prescription refill, scheduling an appointment, reaching someone in the practice with a question, or obtaining lab results. Targets for improvement were increasing the rates for LDL cholesterol or systolic blood pressure at goal, improving patients' physical activity, patient education, and accuracy of risk assessment. Improvement strategies included implementing chart forms, patient education, or care management processes. LIMITATIONS: Patients and charts were selected by physicians reporting their performance for the purpose of MOC. CONCLUSIONS: The Preventive Cardiology PIM successfully provides a self-assessment of practice performance and provides guidance in helping physicians initiate a cycle of quality improvement in their practices.

A three-part model for measuring diabetes care in physician practice.

BACKGROUND: To assess the psychometric properties of the three components of the Diabetes Practice Improvement Module, to compare reliabilities of composites to individual measures, and to identify associations among practice-based and patient-based measures. METHOD: Data include practice systems surveys of 626 physicians, 13,965 chart audits, and 12,927 patient surveys. Quality composites were identified using factor analysis. Means with reliabilities (intraclass correlation coefficient [ICC] and Cronbach's alpha) are reported. Associations among patient-based quality measures and practice measures with case-mix adjustments were estimated via hierarchical models. RESULTS: Composite ICCs range from 0.11 to 0.54, and single items range from 0.05 to 0.49. Staff communication, efficiency, care access, and patient knowledge correlate with patient satisfaction (P < .001). Clinical outcomes are associated with clinical processes (e.g., annual foot exam) and appropriate treatment (P < .001). Patient adjusters (e.g., overall health or factors limiting self-care) are important for the models; physician characteristics used (e.g., age, practice size) seem less important. CONCLUSIONS: Composites require smaller patient sample sizes and result in more reliable measures than do individual items. Additionally, the data show meaningful relationships between composites; physician-directed components (i.e., clinical processes and treatments) are related to clinical outcomes, and patients are clearly more satisfied with care if it is easily accessible and if communication about care is good.