Assessing Physician Resident Contributions to Outpatient Clinical Workload.

BACKGROUND: Graduate medical education is centered in hospitals despite a care system where patients mostly receive their care in an outpatient setting. Such gaps may exist because of inadequate funding for residency positions in community and hospital-based clinics. OBJECTIVE: Determine if physician residents' contribution to outpatient workload offsets their costs for supervision, salary, and fringe benefits as residents acquire skills to become independent practitioners. RESEARCH DESIGN: VA's electronic patient records from 2005 through 2018 were analyzed using generalized linear mixed models to estimate resident and staff contributions to workload in relative value units. MEASURES: Resident participation rate is resident contributed workload net of supervision as a percent of total clinic workload. Productivity is per diem resident workload as a percent of per diem staff workload. Efficiency is per dollar resident workload as a percent of per dollar staff workload. Progressive independence is annual rate of change in resident productivity. RESULTS: Average participation rates varied by specialty from 6% to 22%, with 11% (primary care) and 13% (psychiatry). Productivity rates ranged from 21% to 94%, with 57% (primary care) and 61% (psychiatry). Efficiency rates varied from 0.63 to 3.81, with 1.69 (primary care), 1.89 (psychiatry). Progressive independence rates varied from 2.7%/year (psychiatry) to 39.7%/year (specialty care). CONCLUSIONS: Although residents rotating through most VA clinics generate revenue to cover their direct costs as they learn, some federal subsidies may be necessary to encourage hospital- and community-based clinics to accept residents from the less profitable primary care and mental health specialties.

Does Psychological Safety Impact the Clinical Learning Environment for Resident Physicians? Results From the VA's Learners' Perceptions Survey.

BACKGROUND: Psychological safety (PS) is the perception that it is safe to take interpersonal risks in the work environment. In teaching hospitals, PS may influence the clinical learning environment for trainees. OBJECTIVE: We assessed whether resident physicians believe they are psychologically safe, and if PS is associated with how they rate satisfaction with their clinical learning experience. METHODS: Data were extracted from the Learners' Perceptions Survey (LPS) of residents who rotated through a Department of Veterans Affairs health care facility for academic years 2011-2014. Predictors of PS and its association with resident satisfaction were adjusted to account for confounding and response rate biases using generalized linear models. RESULTS: The 13 044 respondents who completed the LPS (30% response rate) were comparable to nonpediatric, non-obstetrics-gynecology residents enrolled in US residency programs. Among respondents, 11 599 (89%) agreed that ". . . members of the clinical team of which I was part are able to bring up problems and tough issues." Residents were more likely to report PS if they were male, were in a less complex clinical facility, in an other medicine or psychiatry specialty, or cared for patients who were aged, had multiple illnesses, or had social supports. Nonpsychiatric residents felt safer when treating patients with no concurrent mental health diagnoses. PS was strongly associated with how residents rated their satisfaction across 4 domains of their clinical learning experience (P < .001). CONCLUSIONS: PS appears to be an important factor in resident satisfaction across 4 domains that evaluators of graduate medical education programs should consider when assessing clinical learning experiences.

A systematic approach to subgroup analyses in a smoking cessation trial.

BACKGROUND: Traditional approaches to subgroup analyses that test each moderating factor as a separate hypothesis can lead to erroneous conclusions due to the problems of multiple comparisons, model misspecification, and multicollinearity. OBJECTIVE: To demonstrate a novel, systematic approach to subgroup analyses that avoids these pitfalls. METHODS: A Best Approximating Model (BAM) approach that identifies multiple moderators and estimates their simultaneous impact on treatment effect sizes was applied to a randomized, controlled, 11-week, double-blind efficacy trial on smoking cessation of adult smokers with attention-deficit/hyperactivity disorder (ADHD), randomized to either OROS-methylphenidate (n = 127) or placebo (n = 128), and treated with nicotine patch. Binary outcomes measures were prolonged smoking abstinence and point prevalence smoking abstinence. RESULTS: Although the original clinical trial data analysis showed no treatment effect on smoking cessation, the BAM analysis showed significant subgroup effects for the primary outcome of prolonged smoking abstinence: (1) lifetime history of substance use disorders (adjusted odds ratio [AOR] 0.27; 95% confidence interval [CI] 0.10-0.74), and (2) more severe ADHD symptoms (baseline score >36; AOR 2.64; 95% CI 1.17-5.96). A significant subgroup effect was also shown for the secondary outcome of point prevalence smoking abstinence--age 18 to 29 years (AOR 0.23; 95% CI 0.07-0.76). CONCLUSIONS: The BAM analysis resulted in different conclusions about subgroup effects compared to a hypothesis-driven approach. By examining moderator independence and avoiding multiple testing, BAMs have the potential to better identify and explain how treatment effects vary across subgroups in heterogeneous patient populations, thus providing better guidance to more effectively match individual patients with specific treatments.

Comparing clinical predictors of deep venous thrombosis versus pulmonary embolus after severe injury: a new paradigm for posttraumatic venous thromboembolism?

BACKGROUND: The traditional paradigm is that deep venous thrombosis (DVT) and pulmonary embolus (PE) are different temporal phases of a single disease process, most often labeled as the composite end point venous thromboembolism (VTE). However, we theorize that after severe blunt injury, DVT and PE may represent independent thrombotic entities rather than different stages of a single pathophysiologic process and therefore exhibit different clinical risk factor profiles. METHODS: We examined a large, multicenter prospective cohort of severely injured blunt trauma patients to compare clinical risk factors for DVT and PE, including indicators of injury severity, shock, resuscitation parameters, comorbidities, and VTE prophylaxis. Independent risk factors for each outcome were determined by cross-validated logistic regression modeling using advanced exhaustive model search procedures. RESULTS: The study cohort consisted of 1,822 severely injured blunt trauma patients (median Injury Severity Score [ISS], 33; median base deficit, -9.5). Incidence of DVT and PE were 5.1% and 3.9%, respectively. Only 9 (5.7%) of 73 patients with a PE were also diagnosed with DVT. Independent risk factors associated with DVT include prophylaxis initiation within 48 hours (odds ratio [OR], 0.57; 95% confidence interval [CI], 0.36-0.90) and thoracic Abbreviated Injury Scale (AIS) score of 3 or greater (OR, 1.82; 95% CI, 1.12-2.95), while independent risk factors for PE were serum lactate of greater than 5 (OR, 2.33; 95% CI, 1.43-3.79) and male sex (OR, 2.12; 95% CI, 1.17-3.84). Both DVT and PE exhibited differing risk factor profiles from the classic composite end point of VTE. CONCLUSION: DVT and PE exhibit differing risk factor profiles following severe injury. Clinical risk factors for diagnosis of DVT after severe blunt trauma include the inability to initiate prompt pharmacologic prophylaxis and severe thoracic injury, which may represent overall injury burden. In contrast, risk factors for PE are male sex and physiologic evidence of severe shock. We hypothesize that postinjury DVT and PE may represent a broad spectrum of pathologic thrombotic processes as opposed to the current conventional wisdom of peripheral thrombosis and subsequent embolus.

Early blood product and crystalloid volume resuscitation: risk association with multiple organ dysfunction after severe blunt traumatic injury.

BACKGROUND: Elements of volume resuscitation from hemorrhagic shock, such as amount of blood product and crystalloid administration, have been shown to be associated with multiple organ dysfunction (MOD). However, it is unknown whether these are causative factors or merely markers of an underlying requirement for large-volume resuscitation. We sought to further delineate the relevance of the major individual components of early volume resuscitation to onset of MOD after severe blunt traumatic injury. METHODS: We performed a secondary analysis of a large, multicenter prospective observational cohort of severely injured blunt trauma patients, the NIGMS Trauma Glue Grant, to assess the relevance of individual components of resuscitation administered in the first 12 hours of resuscitation including packed red blood cells (PRBC), fresh frozen plasma (FFP), and isotonic crystalloid, to the onset of MOD within the first 28 days after injury. Deaths within 48 hours of injury were excluded. We used a two tiered, exhaustive logistic regression model search technique to adjust for potential confounders from clinically relevant MOD covariates, including indicators of shock severity, injury severity, comorbidities, age, and gender. RESULTS: The study cohort consisted of 1,366 severely injured blunt trauma patients (median new Injury Severity Score = 34). Incidence of 28-day Marshall MOD was 19.6%. Transfusion of ≥10 Units of PRBC in the first 12 hours (odds ratio, 2.06; 95% confidence interval 1.44-2.94), but not FFP (≥8 U) or large volume crystalloid administration (≥12 L), was independently associated with onset of 28-day Marshall MOD. PRBC:FFP ratio in the first 12 hours was not significantly associated with MOD. CONCLUSIONS: When controlling for all major components of acute volume resuscitation, massive-transfusion volumes of PRBC's within the first 12 hours of resuscitation are modestly associated with MOD, whereas FFP and large volume crystalloid administration are not independently associated with MOD. Previous reported associations of blood products and large-volume crystalloid with MOD may be reflecting overall resuscitation requirements and burden of injury rather than independent causation.

Exercise as an augmentation treatment for nonremitted major depressive disorder: a randomized, parallel dose comparison.

OBJECTIVE: Most patients with major depressive disorder (MDD) require second-step treatments to achieve remission. The Treatment with Exercise Augmentation for Depression (TREAD) study was designed to test the efficacy of aerobic exercise as an augmentation treatment for MDD patients who had not remitted with antidepressant treatment. METHOD: Eligible participants in this randomized controlled trial were sedentary individuals (men and women aged 18-70 years) diagnosed with DSM-IV nonpsychotic MDD who had not remitted with selective serotonin reuptake inhibitor (SSRI) treatment. Participants were recruited through physician referrals and advertisements. A total of 126 participants were randomized to augmentation treatment with either 16 kcal per kg per week (KKW) or 4 KKW of exercise expenditure for 12 weeks while SSRI treatment was held constant. Supervised sessions were conducted at The Cooper Institute, Dallas, Texas, with additional home-based sessions as needed to fulfill the weekly exercise prescription. The primary outcome was remission (as determined by a score ≤ 12 on the Inventory of Depressive Symptomatology, Clinician-Rated). The study took place between August 2003 and August 2007. RESULTS: There were significant improvements over time for both groups combined (F1,121 = 39.9, P < .0001), without differential group effect (group effect: F1,134 = 3.2, P = .07; group-by-time effect: F1,119 = 3.8, P = .06). Adjusted remission rates at week 12 were 28.3% versus 15.5% for the 16-KKW and 4-KKW groups, respectively, leading to a number needed to treat (NNT) of 7.8 for 16 KKW versus 4 KKW. Men, regardless of family history of mental illness, and women without a family history of mental illness had higher remission rates by week 12 with higher-dose (women, 39.0%; men, 85.4%) than with lower-dose exercise (women, 5.6%; men, 0.1%) (women: t95 = 2.1, P = .04; men: t88 = 5.4, P < .0001) (NNT: women, 3.0; men, 1.2). CONCLUSIONS: There was a trend for higher remission rates in the higher-dose exercise group (P < .06), with a clinically meaningful NNT of 7.8 in favor of the high exercise dose. Significant differences between groups were found when the moderating effects of gender and family history of mental illness were taken into account and suggest that higher-dose exercise may be better for all men and for women without a family history of mental illness. TRIAL REGISTRATION: clinicaltrials.gov Identifier: NCT00076258.

Studying the effects of ACGME duty hours limits on resident satisfaction: results from VA learners' perceptions survey.

BACKGROUND: As the Accreditation Council on Graduate Medical Education (ACGME) deliberates over further limiting duty hours of graduate medical education (GME) trainees, few large-scale studies have shown residents to be satisfied with the effect the 2003 standards have had on clinical care, education outcomes, or working environments. This study measures the effect of the 2003 duty hours limits on resident-reported satisfaction with GME training during their rotations through the Department of Veterans Affairs (VA) medical centers from 2001 through 2007. METHOD: Self-reported satisfaction with clinical care and education environments were assessed by comparing responses to VA's annual Learners' Perceptions Survey administered before 2003 with responses administered after 2003. To measure duty hours effects on satisfaction, before-after differences were adjusted for covariate biases modeled after an exhaustive covariate search with 10-fold cross-validation. Because nonteaching controls are not available in satisfaction studies, we used a robust differencing variable technique to control before-after differences for trend biases in the simultaneous presence of missing data and possible model misspecification. RESULTS: There were 19,605 responders. Adjusting for covariate and trend biases, after the 2003 ACGME standards, 25% more residents in medicine specialties reported satisfaction with VA clinical environment and 11% more with VA preceptors and faculty. For surgery, 33% more residents reported satisfaction with VA clinical environment and 12% more with VA preceptors and faculty. Satisfaction with working environment was mixed. CONCLUSIONS: The 2003 ACGME duty hours standards were associated with improved satisfaction for resident clinical training and learning environments.

Measuring the intensity of resident supervision in the department of veterans affairs: the resident supervision index.

PURPOSE: To develop a survey instrument designed to quantify supervision by attending physicians in nonprocedural care and to assess the instrument's feasibility and reliability. METHOD: In 2008, the Department of Veterans Affairs (VA) Office of Academic Affiliations convened an expert panel to adopt a working definition of attending supervision in nonprocedural patient care and to construct a survey to quantify it. Feasibility was field-tested on residents and their supervising attending physicians at primary care internal medicine clinics at the VA Loma Linda Healthcare System in their encounters with randomly selected outpatients diagnosed with either major depressive disorder or diabetes. The authors assessed both interrater concurrent reliability and test-retest reliability. RESULTS: The expert panel adopted the VA's definition of resident supervision and developed the Resident Supervision Index (RSI) to measure supervision in terms of residents' case understanding, attending physicians' contributions to patient care through feedback to the resident, and attending physicians' time (minutes). The RSI was field-tested on 60 residents and 37 attending physicians for 148 supervision episodes from 143 patient encounters. Consent rates were 94% for residents and 97% for attending physicians; test-retest reliability intraclass correlations (ICCs) were 0.93 and 0.88, respectively. Concurrent reliability between residents' and attending physicians' reported time was an ICC of 0.69. CONCLUSIONS: The RSI is a feasible and reliable measure of resident supervision that is intended for research studies in graduate medical education focusing on education outcomes, as well as studies assessing quality of care, patient health outcomes, care costs, and clinical workload.

Measuring progressive independence with the resident supervision index: theoretical approach.

BACKGROUND: Graduate medical education is based on an on-the-job training model in which residents provide clinical care under supervision. The traditional method is to offer residents graduated levels of responsibility that will prepare them for independent practice. However, if progressive independence from supervision exceeds residents' progressive professional development, patient outcomes may be at risk. Leaders in graduate medical education have called for "optimal" supervision, yet few studies have conceptually defined what optimal supervision means and whether optimal care is theoretically compatible with progressive independence, nor have they developed a test for progressive independence. OBJECTIVE: This research develops theory and analytic models as part of the Resident Supervision Index to quantify the intensity of supervision. METHODS: We introduce an explicit set of assumptions for an ideal patient-centered theory of optimal supervision of resident-provided care. A critical assumption is that informed attending staff will use available resources to optimize patient outcomes first and foremost, with residents gaining clinical competencies by contributing to optimal care. Next, we derive mathematically the consequences of these assumptions as theoretical results. RESULTS: Under optimal supervision, (1) patient outcome is expected to be no worse than if residents were not involved, (2) supervisors will avoid undersupervising residents (when patients are at increased risk for poor outcomes) or oversupervising residents (when residents miss clinical opportunities to practice care), (3) optimal patient outcomes will be compatible with progressive independence, (4) progressive development can be inferred from progressive independence whenever residents contribute to patient care, and (5) analytic models that test for progressive independence will emphasize adjusting the association between length of graduate medical education training and supervision for case complexity and clinic workload, but not patient health outcomes. CONCLUSION: An explicit theoretical framework is critical to measure scientifically progressive independence from supervision using graduate medical education data.

Measuring progressive independence with the resident supervision index: empirical approach.

BACKGROUND: A Resident Supervision Index (RSI) developed by our research team quantifies the intensity of resident supervision in graduate medical education, with the goal of testing for progressive independence. The 4-part RSI method includes a survey instrument for staff and residents (RSI Inventory), a strategy to score survey responses, a theoretical framework (patient centered optimal supervision), and a statistical model that accounts for the presence or absence of supervision and the intensity of patient care. METHODS: The RSI Inventory data came from 140 outpatient encounters involving 57 residents and 37 attending physicians during a 3-month period at a Department of Veterans Affairs outpatient clinic. Responses are scored to quantitatively measure the intensity of resident supervision across 10 levels of patient services (staff is absent, is present, participated, or provided care with or without a resident), case discussion (resident-staff interaction), and oversight (staff reviewed case, reviewed medical chart, consulted with staff, or assessed patient). Scores are analyzed by level and for patient care using a 2-part model (supervision initiated [yes or no] versus intensity once supervision was initiated). RESULTS: All resident encounters had patient care supervision, resident oversight, or both. Consistent with the progressive independence hypothesis, residents were 1.72 (P  =  .019) times more likely to be fully responsible for patient care with each additional postgraduate year. Decreasing case complexity, increasing clinic workload, and advanced nonmedical degrees among attending staff were negatively associated with supervision intensity, although associations varied by supervision level. CONCLUSIONS: These data are consistent with the progressive independence hypothesis in graduate medical education and offer empirical support for the 4-part RSI method to quantify the intensity of resident supervision for research, program evaluation, and resident assessment purposes. Before informing policy, however, more scientific research in actual teaching settings is needed to better understand the relationships among patient outcomes, clinic workload, case complexity, and graduate medical education experience in resident supervision and professional development.

Measuring use and cost of care for patients with mood disorders: the utilization and cost inventory.

BACKGROUND: Researchers conducting cost-outcome studies must account for all materially relevant care that subjects receive from their care providers. However, access to provider records is often limited. This article describes and tests the Utilization and Cost Inventory (UAC-I), a structured patient interview designed to measure costs of care when access to provider records is limited. METHODS: UAC-I was tested on 212 consenting adult veterans with mood disorder attending a VA medical center. Counts (inpatient days and outpatient encounters) and costs (dollars) computed from survey responses were compared with estimates from medical records and an alternative structured questionnaire. RESULTS: The agreement between inpatient costs computed from provider records and from UAC-I responses, assessed using the intraclass correlation coefficient (ICC), was 0.66, 95% confidence interval (CI), 0.30-0.84; the bias was -3.7%, 95% CI, -48 to 41. The ICC for the service data (inpatient days) was 0.97, 95% CI, 0.95-0.99; the bias was <1%, 95% CI, -14 to 15. The ICC for outpatient costs computed from provider records and from UAC-I responses was 0.53 95% CI, 0.38-0.65; the bias was <1%, 95% CI, -27 to 27. The ICC for outpatient encounters was 0.74, 95% CI, 0.65-0.80; the bias was <1%, 95% CI, -16 to 18. CONCLUSIONS: These results indicate that it may be feasible for cost-outcome studies to compare patient groups for inpatient and outpatient costs computed from patient self-reports.

Assessing the preventive effects of cognitive therapy following relief of depression: A methodological innovation.

BACKGROUND: Strategies to compute benefits from continuing cognitive therapy for patients with recurrent major depression do not take into account whether discontinuing treatments may induce temporary increases in the risk that symptoms return (discontinuation-effect). METHODS: We apply varying-effects analyses and compare findings with traditional methods to assess the effects of continuation-phase cognitive therapy. Two years of data came from 79 patients with recurrent major depression who responded to acute cognitive therapy. Patients were randomized to either an experimental cohort (n=39) who received 10-session, protocol continuation-phase therapy for 8 months, or a control cohort (n=40) who stopped protocol treatment after the acute-phase. Symptoms were assessed using the Longitudinal Interval Follow-up Evaluation (LIFE). Symptom risk rates were computed weekly by cohort as the proportion of patients at risk who were suffering from a major depressive episode. RESULTS: Significant discontinuation-effects occurred when protocol treatments stopped for both experimental and control cohorts. Following acute-phase care, traditional computation methods (week 1-35) revealed treated patients had 18% of the risk for symptoms as controls. Expanding the observation period (week 1-74) to include these discontinuation-effects revealed more modest initial effect sizes (43%), but significant long-term effects (54% for week 75-101). LIMITATIONS: Limitations include limited sample size, one-site study, confounds from patient-level interactions, and off-protocol use of depression-related care. CONCLUSIONS: Varying-effects analyses can describe how outcomes from cognitive therapy may unfold over time for patients with major depression. These analyses reveal complex longitudinal patterns that otherwise are not detectable with traditional time-to-event methods. Specifically, we observed discontinuation-effects, or temporary spikes in symptom risks that occur after treatment ends. Further research is needed to identify the mechanisms driving these effects. Future studies are needed to determine if higher risks result from the patients' anxiety as they attempt to maintain gains independent of ongoing therapy, or reflect residual symptoms previously suppressed by treatment. We also observed longer-term preventive effects from therapy. Again, further research is recommended to determine the extent to which lower risks result from coping and compensatory skills learned during cognitive therapy. These findings suggest that varying-effects analyses may provide an appealing paradigm for understanding treatment-related effects in episodic illness.

A data accounting system for clinical investigators.

Clinical investigators often preprocess, process, and analyze their data without benefit of formally organized research centers to oversee data management. This article outlines a practical three-file structure to help guide these investigators track and document their data through processing and analyses. The proposed process can be implemented without additional training or specialized software. Thus, it is particularly well suited for research projects with small budgets or limited access to viable research/data coordinating centers.