An Analysis of Written and Numeric Scores in End-of-Rotation Forms from Three Residency Programs.

Introduction: End-of-Rotation Forms (EORFs) assess resident progress in graduate medical education and are a major component of Clinical Competency Committee (CCC) discussion. Single-institution studies suggest EORFs can detect deficiencies, but both grades and comments skew positive. In this study, we sought to determine whether the EORFs from three programs, including multiple specialties and institutions, produced useful information for residents, program directors, and CCCs. Methods: Evaluations from three programs were included (Program 1, Institution A, Internal Medicine: n = 38; Program 2, Institution A, Anesthesia: n = 9; Program 3, Institution B, Anesthesia: n = 11). Two independent researchers coded each written comment for relevance (specificity and actionability) and orientation (praise or critical) using a standardized rubric. Numeric scores were analyzed using descriptive statistics. Results: 4869 evaluations were collected from the programs. Of the 77,434 discrete numeric scores, 691 (0.89%) were considered "below expected level." 71.2% (2683/3767) of the total written comments were scored as irrelevant, while 3217 (85.4%) of total comments were scored positive and 550 (14.6%) were critical. When combined, 63.2% (n = 2379) of comments were scored positive and irrelevant while 6.5% (n = 246) were scored critical and relevant. Discussion: <1% of comments indicated below average performance; >70% of comments scored irrelevant. Critical, relevant comments were least frequently observed, consistent across all 3 programs. The low rate of constructive feedback and the high rate of irrelevant comments are inadequate for a CCC to make informed decisions. The consistency of these findings across programs, specialties, and institutions suggests both local and systemic changes should be considered.

Immune Regulatory Cell Bias Following Alemtuzumab Treatment in Relapsing-Remitting Multiple Sclerosis.

Alemtuzumab is a highly effective treatment for relapsing-remitting multiple sclerosis. It selectively targets the CD52 antigen to induce profound lymphocyte depletion, followed by recovery of T and B cells with regulatory phenotypes. We previously showed that regulatory T cell function is restored with cellular repletion, but little is known about the functional capacity of regulatory B-cells and peripheral blood monocytes during the repletion phase. In this study (ClinicalTrials.gov ID# NCT03647722) we simultaneously analyzed the change in composition and function of both regulatory lymphocyte populations and distinct monocyte subsets in cross-sectional cohorts of MS patients prior to or 6, 12, 18, 24 or 36 months after their first course of alemtuzumab treatment. We found that the absolute number and percentage of cells with a regulatory B cell phenotype were significantly higher after treatment and were positivity correlated with regulatory T cells. In addition, B cells from treated patients secreted higher levels of IL-10 and BDNF, and inhibited the proliferation of autologous CD4+CD25- T cell targets. Though there was little change in monocytes populations overall, following the second annual course of treatment, CD14+ monocytes had a significantly increased anti-inflammatory bias in cytokine secretion patterns. These results confirmed that the immune system in alemtuzumab-treated patients is altered in favor of a regulatory milieu that involves expansion and increased functionality of multiple regulatory populations including B cells, T cells and monocytes. Here, we showed for the first time that functionally competent regulatory B cells re-appear with similar kinetics to that of regulatory T-cells, whereas the change in anti-inflammatory bias of monocytes does not occur until after the second treatment course. These findings justify future studies of all regulatory cell types following alemtuzumab treatment to reveal further insights into mechanisms of drug action, and to identify key immunological predictors of durable clinical efficacy in alemtuzumab-treated patients.