Innovation Reports: Successes and Limitations for Promoting Innovation in Medical Education.

In 2013, Academic Medicine introduced a new article type, Innovation Reports, with the intent to promote innovation by providing a forum for publishing promising new ideas at an early stage of development. In this article, the authors examine Innovation Reports as a means for promoting innovation within the medical education community.The authors undertook a 2-part analysis. In July 2018, they examined submission data, Altmetric scores, and citations for the first 5 years of the feature. To explore authors' perceptions of the impact of publishing an Innovation Report, in October 2018, they conducted a web-based survey of corresponding authors.Between October 2013 and May 2018, 920 manuscripts were submitted under the Innovation Report category, of which 335 were sent for review and 151 were published (16% overall acceptance rate). The mean citation rate for the published Innovation Reports was 4.3 (range 0-47). The mean Altmetric Attention Score was 14.3 (range 0-441). An Altmetric score of 14 places an article in the top 20% of articles published in Academic Medicine.The 151 published Innovation Reports had 148 unique corresponding authors, of whom 85 (57%) responded to the survey. Almost all respondents (n = 82; 96%) reported that publishing an Innovation Report promoted their individual career growth. For many corresponding authors, the publication of early ideas in an Innovation Report appeared to be an end point rather than a springboard for further development and innovation, as only 14 (16%) reported publication of a subsequent related study.Reflecting on the successes and limitations of Innovation Reports over the first 5 years, the authors suggest that soliciting input from more stakeholders and being explicit about the goals of this article type would help inform how Innovation Reports should evolve in the future so they can better launch creative thought and spur innovation.

A Multi-Institution Collaboration to Define Core Content and Design Flexible Curricular Components for a Foundational Medical School Course: Implications for National Curriculum Reform.

Medical educators have not reached widespread agreement on core content for a U.S. medical school curriculum. As a first step toward addressing this, five U.S. medical schools formed the Robert Wood Johnson Foundation Reimagining Medical Education collaborative to define, create, implement, and freely share core content for a foundational medical school course on microbiology and immunology. This proof-of-concept project involved delivery of core content to preclinical medical students through online videos and class-time interactions between students and facilitators. A flexible, modular design allowed four of the medical schools to successfully implement the content modules in diverse curricular settings. Compared with the prior year, student satisfaction ratings after implementation were comparable or showed a statistically significant improvement. Students who took this course at a time point in their training similar to when the USMLE Step 1 reference group took Step 1 earned equivalent scores on National Board of Medical Examiners-Customized Assessment Services microbiology exam items. Exam scores for three schools ranged from 0.82 to 0.84, compared with 0.81 for the national reference group; exam scores were 0.70 at the fourth school, where students took the exam in their first quarter, two years earlier than the reference group. This project demonstrates that core content for a foundational medical school course can be defined, created, and used by multiple medical schools without compromising student satisfaction or knowledge. This project offers one approach to collaboratively defining core content and designing curricular resources for preclinical medical school education that can be shared.

Practice Implications of Expanded Genetic Testing in Oncology.

Genetic test use in oncology is growing, yet providers' experiences with evolving testing norms and their implications for patient care remain under-explored. In interviews with oncologists and cancer genetics professionals, 22 key informants described the increasing importance of germline results for therapeutic decision-making, preference for ordering tests directly rather than referring, and rapid adoption of cancer gene panels for testing. Implications for informed consent, result interpretation, and patient management were identified. These results suggest concerns raised by the transition of genetic test delivery from cancer genetics professionals to oncologists that must be addressed in practice guidelines and provider training.

A Survey of Perceived Effectiveness of Part 4 Maintenance of Certification.

OBJECTIVES: Evaluative assessment is needed to inform improvement of Part 4 Maintenance of Certification (MOC), a large-scale program that aims to improve physician knowledge, engagement, and skills in quality improvement (QI). We sought to determine if Part 4 MOC participation improves perceived educational and clinical outcomes by piloting a new physician survey. METHODS: We administered a new online survey (MOC Practice, Engagement, Attitude, and Knowledge Survey) to physicians at the beginning and end of a Part 4 MOC project sponsored by a pediatric hospital's American Board of Medical Specialties' portfolio program during 2015. Participants worked in academic and community settings and in various accredited specialties. The main outcome was change in survey response on a 5-point Likert scale (1 = best) for 3 learning domains (QI engagement and attitude; QI method application, and improved patient care). RESULTS: Of 123 complete responses and a 97% response rate, mean baseline responses were positive or neutral (2.2, 2.3, 1.9, respectively). Responses improved in QI engagement and attitude (-0.15, z score = -2.78, P = .005), QI method application (-0.39, z score = -7.364, P < .005), and improved patient care (-0.11, z score = -1.728, P = .084). CONCLUSIONS: A Part 4 MOC physician survey provides valuable data to evaluate and improve the learning activity. In this children's hospital program, physicians view Part 4 favorably. Participation was associated with modest improvements in perceptions of QI engagement and attitude, application of QI methods, and patient care. Systematic evaluation of all Part 4 MOC projects and programs has the potential to improve the program nationally.

Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research.

The authors of this article reviewed the methodology of three common consensus methods: nominal group process, consensus development panels, and the Delphi technique. The authors set out to determine how a majority of researchers are conducting these studies, how they are analyzing results, and subsequently the manner in which they are reporting their findings. The authors conclude with a set of guidelines and suggestions designed to aid researchers who choose to use the consensus methodology in their work.Overall, researchers need to describe their inclusion criteria. In addition to this, on the basis of the current literature the authors found that a panel size of 5 to 11 members was most beneficial across all consensus methods described. Lastly, the authors agreed that the statistical analyses done in consensus method studies should be as rigorous as possible and that the predetermined definition of consensus must be included in the ultimate manuscript. More specific recommendations are given for each of the three consensus methods described in the article.

Using the Patient Centered Observation Form: Evaluation of an online training program.

OBJECTIVE: The Patient Centered Observation Form (PCOF) helps trainees identify and describe specific communication skills and enhance self-awareness about skill use. We studied the effectiveness and ease of use of the Improving Communication Assessment Program (ICAP), an online module that prepares trainees to use the PCOF. METHODS: Students, residents and medical educators viewed two videos (common and better skill use) of the same interaction and rated each video using the PCOF. Video sequence was randomized. We assessed agreement with experts, ease of use, concepts learned, and areas of confusion. RESULTS: Trainees (211) achieved strong agreement (.83) with experts and were highly satisfied (mean 4.18 out of 5). Viewing the common video first produced higher agreement (.87 vs .79; ES=.4) with experts and greater satisfaction (4.36 vs 4.02, ES .4) than viewing the better video first. Trainees reported diverse areas of learning and minimal confusion. CONCLUSION: ICAP training to use the PCOF may facilitate teaching and assessment of communication skills and enrich training through peer observation and feedback. We offer several educational strategies. PRACTICE IMPLICATIONS: Learning to use the PCOF via the ICAP module may accelerate communication training for medical students, residents, medical educators and practicing clinicians.

Training process of American family medicine and their enlightenment on China / 中华医学教育探索杂志

General practitioner system is an important part of national health service system.Improving general medical education is an inevitable tendency of modern medical education.General medical science starts late in China.Development of Chinese general practitioner training system and general medical education are relatively insufficient and lack of relevant experiences.This paper introduced family doctor training process of Washington university school of medicine and revealed characteristics of systematization,standardization and pervasive of it to inspire our country to learn from the following aspects:(1) paying fully attention to the important role of general medical science in the national health service system； (2) systematically planning three-step training of general medical education ； (3) implementing general medical education teaching content and teaching staff development.

Where do medical students turn? The role of the assigned mentor in the fabric of support during medical school.

BACKGROUND: The University of Washington School of Medicine implemented an assigned mentoring program in 2002. The College Mentors are assigned at matriculation, advise students throughout medical school, and teach and evaluate students in the 2nd-year Introduction to Clinical Medicine course. PURPOSE: The purpose of the study was to determine from whom students report they would seek advice and support for academic, professional, personal, and research issues. METHODS: A cross-sectional cohort survey asking students whom they would first contact about academic, personal, professional, and research issues was administered to three cohorts of students in 2007. RESULTS: Students reported that they would contact their College Mentor first for general academic progress (49.6%), personal issues (36.2%), and professional issues (64.1%) but not for research issues. CONCLUSIONS: Students identified their College Mentor as a primary contact for academic, professional, and personal issues, suggesting that neither the mentors' assigned status or evaluator role were barriers to the mentoring relationship.

A survey on financial support and research achievement of medical education research units in China.

AIM: To investigate the current situation of financial support and research achievement of medical education research units in China. METHODS: A total of 46 individuals in 46 medical schools completed a questionnaire including information about affiliation of the unit, financial support, published articles and achievement awards of the units. RESULT: Of the 46 schools, 24 had independent medical education research units, 36 had financial support, and 30 had research funding. The mean number of published articles was 2.53 per staff. The mean number of achievement awards was 3.80 per unit. There was a significant difference in funding and published articles between independent medical education research units and other types of units; and in published articles and achievement awards between the units with funding and without funding. CONCLUSION: The financial support from the school was the main source of medical education research units in China. More attention should be paid to the development of medical education research units, to their ability to produce high quality research and support the improvement of medical care in China.

Measuring the quality of end-of-life care.

CONTEXT: Although there is a documented need to improve end-of-life care, there are few validated and brief questionnaires that are available as outcome measures for use in improving that care. OBJECTIVES: To examine the measurement characteristics of the Quality of End-of-Life Care (QEOLC) questionnaire. METHODS: In a multisite, cross-sectional study of a mailed questionnaire, patients with life-limiting illnesses, their families, and nurses completed the QEOLC questionnaire. Patients and nurses were identified by physicians, and families were identified by participating patients. Physicians included general internists, oncologists, cardiologists, and pulmonologists from the Southeast and Pacific Northwest of the United States. RESULTS: Eight hundred one patients, 310 of their families, and 885 nurses were identified by 85 physicians. Using structural equation modeling techniques corrected for clustering under physicians, we identified a patient-specific factor based on 11 items, a family-specific factor based on 22 items, a nurse-specific factor based on 11 items, and a common single-factor solution based on 10 items. Construct validity was supported by significant associations in the hypothesized direction between the identified QEOLC factors and each of the following: physician palliative care knowledge, patients' and families' ratings of overall quality of care, and patients' levels of symptom distress. CONCLUSION: Although continued testing in heterogeneous samples is necessary, the current study supported the construct validity of the QEOLC questionnaire to assess physician skill at end-of-life care, thereby providing valid measures of quality end-of-life care. Furthermore, this approach is a model for development and validation of patient- and family-centered assessments of quality of care.

Defining and describing medical learning communities: results of a national survey.

PURPOSE: To investigate what is meant by learning community in medical education and to identify the most important features of current medical education learning communities. METHOD: After a literature review, the authors surveyed academic deans of all U.S. and Canadian medical schools and colleges (N=124) to identify those that had implemented a learning community. Those with student learning communities (N=18) answered a series of questions about the goals, structure, function, benefits, and challenges of their communities. RESULTS: The most common primary goals included fostering communication among students and faculty; promoting caring, trust, and teamwork; helping students establish academic support networks; and helping students establish social support networks. Most deans said that students remained in the same community for all four years of medical school and that communities were linked to specific faculty and/or peer advisors. For most schools, communities included students from many class years, and participation was mandatory. Curricular purposes included professionalism training, leadership development, and service learning. Almost all schools had social functions related to their communities, and most provided career planning, group mentoring, and personal counseling. CONCLUSIONS: Learning communities in medical education demonstrate diverse approaches to achieving the general goal of enhanced student learning. Medical school leaders considering learning communities should determine the goals they want to accomplish and be open to adopting different approaches based on local needs. Evaluation and effective monitoring of evolution are needed to determine the best approaches for different needs and to assess impact on students and faculty.

Pilot study of a 'RIME'-based tool for giving feedback in a multi-specialty longitudinal clerkship.

CONTEXT: The Reporter-Interpreter-Manager-Educator (RIME) evaluation framework is intuitive and reliable. Our preceptors' frustration with using summative tools for formative feedback and the hypothesis that the RIME vocabulary might improve students' and preceptors' experiences with feedback prompted us to develop and pilot a RIME-based feedback tool. METHODS: The tool was based on the RIME vocabulary, which has previously been used for evaluation. As interpersonal skills and professionalism are difficult areas in which to give feedback, we added these as explicit categories. We piloted the tool in a longitudinal, 5-month, multi-specialty clerkship. Preceptors completed pre- and post-introductory workshop surveys. Students completed post-workshop and post-clerkship surveys. RESULTS: Preceptors (n = 14) and students (n = 8) preferred RIME-based feedback to 'usual feedback' (previously given using end-of-clerkship evaluation forms). After the initial workshop, preceptors expected that giving feedback, including critical feedback, would be easier. After the 5-month clerkship, students reported receiving more feedback than in previous clerkships and rated feedback given using this tool more highly (P = 0.002; effect size 1.2). Students also felt it helped them understand specifically how to improve their performance (P = 0.003; effect size 1.2). DISCUSSION: In this pilot study, preceptors and students preferred feedback with a specific RIME-based tool. Students felt such feedback was more useful and helped them identify specifically how to improve. Whether this method can improve student performance through improved feedback remains an area for further research.

Crafting Successful Relationships with the IRB.

BACKGROUND: Medical educational researchers face frustration with IRBs for activities that formerly were exempt from review or were not treated as biomedical research with patients. The authors sought to identify methods for improving relationships between IRBs and medical education researchers. METHOD: The authors conducted interviews with medical school representatives about factors leading to relationships in which all parties feel that their concerns are being met, subjects are appropriately protected, and that the progress of evaluation or research activities is not unnecessarily inhibited. RESULTS: Successful relationships require efforts at education of the IRB and the researchers. All institutions acknowledged the need to establish and maintain good communication. Some schools developed structures or procedures that resulted in more rapid review and increased satisfaction that interests of all parties were protected. CONCLUSIONS: A relationship must be crafted between medical education researchers and the IRB. The authors found key elements to successful approaches.

Reliability of a 360-degree evaluation to assess resident competence.

OBJECTIVE: To determine the feasibility and psychometric qualities of a 360-degree evaluation of physical medicine and rehabilitation (PM&R) residents' competence. DESIGN: Nurses, allied health staff, and medical students completed a 12-item questionnaire after each PM&R resident rotation from January 2002 to December 2004. The items were derived from five of the six competencies defined by the Accreditation Council for Graduate Medical Education (ACGME). RESULTS: Nine hundred thirty evaluations of 56 residents were completed. The alpha reliability coefficient for the instrument was 0.89. Ratings did not vary significantly by resident gender. Senior residents had higher ratings than junior residents. A reliability of >0.8 could be achieved by ratings from just five nurses or allied health staff, compared with 23 ratings from medical students. Factor analysis revealed all items clustered on one factor, accounting for 84% of the variance. In a subgroup of residents with low scores, raters were able to differentiate among skills. CONCLUSION: Resident assessment tools should be valid, reliable, and feasible. This Web-based 360-degree evaluation tool is a feasible way to obtain reliable ratings from rehabilitation staff about resident behaviors. The assignment of higher ratings for senior residents than junior residents is evidence for the general validity of this 360-degree evaluation tool in the assessment of resident performance. Different rater groups may need distinct instruments based on the exposure of rater groups to various resident activities and behaviors.

Competing professionalism values among community-based family physicians.

BACKGROUND: Professionalism has received considerable attention in recent years, mostly within academic settings. Little attention has been given to the perspectives of practicing physicians on professionalism. This study was designed to determine whether prevailing definitions of and guidelines for professionalism accurately reflect the perspectives and experiences of practicing community-based family physicians. METHOD: Eighteen practicing family physicians in Seattle were recruited from nonacademic settings to participate in focus groups during 2003. Transcripts were analyzed for content themes related to professionalism. RESULTS: Analysis revealed several tensions among values that the physician participants described balancing in their practices. Balancing organizational expectations and individual physician or patient values was a major source of tension. CONCLUSIONS: Practicing family physicians describe various tensions as they attempt to balance their own values with competing values of their patients, the organizations for which many work, and stated values of their profession. Such tensions among values have seldom been described in past literature and should be considered in designing medical curricula at all levels.