Evaluating the Role of Competency-Based Behavioral Interviewing in Holistic Medical School Admissions.

PROBLEM: Holistic review is a multifaceted concept that aims to increase diversity and applicant fit with program needs by complementing traditional academic requirements with appraisal of a wider range of personal characteristics and experiences. Behavioral interviewing has been practiced and studied in human resources, business, and organizational psychology for over 50 years. Its premise is that future performance can be anticipated from past actions. However, many of the interview approaches within the holistic framework are resource intensive and logistically challenging. APPROACH: The Vanderbilt University School of Medicine instituted a competency-based behavioral interview (CBBI) to augment the selection process in 2012. Behavioral interviews are based on key competencies needed for entering students and require applicants to reflect on their actual experiences and what they learned from them. The authors reviewed 5 years of experience (2015-2019) to evaluate how CBBI scores contributed to the overall assessment of applicants for admission. OUTCOMES: The final admission committee decision for each applicant was determined by reviewing multiple factors, with no single assessment determining the final score. The CBBI and summary interview scores showed a strong association (P < .005), suggesting that the summary interviewer, who had access to the full applicant file, and the CBBI interviewer, who did not, assessed similar strengths despite the 2 different approaches, or that the strengths assessed tracked in the same direction. Students whose 2 interview scores were not aligned were less likely to be accepted to the school. NEXT STEPS: The review raised awareness about the cultural aspects of interpreting the competencies and the need to expand our cultural framework throughout interviewer training. Findings indicate that CBBIs have the potential to reduce bias related to over-reliance on standardized metrics; however, additional innovation and research are needed.

Competencies for the Use of Artificial Intelligence-Based Tools by Health Care Professionals.

PURPOSE: The expanded use of clinical tools that incorporate artificial intelligence (AI) methods has generated calls for specific competencies for effective and ethical use. This qualitative study used expert interviews to define AI-related clinical competencies for health care professionals. METHOD: In 2021, a multidisciplinary team interviewed 15 experts in the use of AI-based tools in health care settings about the clinical competencies health care professionals need to work effectively with such tools. Transcripts of the semistructured interviews were coded and thematically analyzed. Draft competency statements were developed and provided to the experts for feedback. The competencies were finalized using a consensus process across the research team. RESULTS: Six competency domain statements and 25 subcompetencies were formulated from the thematic analysis. The competency domain statements are: (1) basic knowledge of AI: explain what AI is and describe its health care applications; (2) social and ethical implications of AI: explain how social, economic, and political systems influence AI-based tools and how these relationships impact justice, equity, and ethics; (3) AI-enhanced clinical encounters: carry out AI-enhanced clinical encounters that integrate diverse sources of information in creating patient-centered care plans; (4) evidence-based evaluation of AI-based tools: evaluate the quality, accuracy, safety, contextual appropriateness, and biases of AI-based tools and their underlying data sets in providing care to patients and populations; (5) workflow analysis for AI-based tools: analyze and adapt to changes in teams, roles, responsibilities, and workflows resulting from implementation of AI-based tools; and (6) practice-based learning and improvement regarding AI-based tools: participate in continuing professional development and practice-based improvement activities related to use of AI tools in health care. CONCLUSIONS: The 6 clinical competencies identified can be used to guide future teaching and learning programs to maximize the potential benefits of AI-based tools and diminish potential harms.

Considering Clinician Competencies for the Implementation of Artificial Intelligence-Based Tools in Health Care: Findings From a Scoping Review.

BACKGROUND: The use of artificial intelligence (AI)-based tools in the care of individual patients and patient populations is rapidly expanding. OBJECTIVE: The aim of this paper is to systematically identify research on provider competencies needed for the use of AI in clinical settings. METHODS: A scoping review was conducted to identify articles published between January 1, 2009, and May 1, 2020, from MEDLINE, CINAHL, and the Cochrane Library databases, using search queries for terms related to health care professionals (eg, medical, nursing, and pharmacy) and their professional development in all phases of clinical education, AI-based tools in all settings of clinical practice, and professional education domains of competencies and performance. Limits were provided for English language, studies on humans with abstracts, and settings in the United States. RESULTS: The searches identified 3476 records, of which 4 met the inclusion criteria. These studies described the use of AI in clinical practice and measured at least one aspect of clinician competence. While many studies measured the performance of the AI-based tool, only 4 measured clinician performance in terms of the knowledge, skills, or attitudes needed to understand and effectively use the new tools being tested. These 4 articles primarily focused on the ability of AI to enhance patient care and clinical decision-making by improving information flow and display, specifically for physicians. CONCLUSIONS: While many research studies were identified that investigate the potential effectiveness of using AI technologies in health care, very few address specific competencies that are needed by clinicians to use them effectively. This highlights a critical gap.

Effect of Smartphone App-Based Education on Clinician Prescribing Habits in a Learning Health Care System: A Randomized Cluster Crossover Trial.

Importance: Effective methods for engaging clinicians in continuing education for learning-based practice improvement remain unknown. Objective: To determine whether a smartphone-based app using spaced education with retrieval practice is an effective method to increase evidence-based practice. Design, Setting, and Participants: A prospective, unblinded, single-center, crossover randomized clinical trial was conducted at a single academic medical center from January 6 to April 24, 2020. Vanderbilt University Medical Center clinicians prescribing intravenous fluids were invited to participate in this study. Interventions: All clinicians received two 4-week education modules: 1 on prescribing intravenous fluids and 1 on prescribing opioid and nonopioid medications (counterbalancing measure), over a 12-week period. The order of delivery was randomized 1:1 such that 1 group received the fluid management module first, followed by the pain management module after a 4-week break, and the other group received the pain management module first, followed by the fluid management module after a 4-week break. Main Outcomes and Measures: The primary outcome was evidence-based clinician prescribing behavior concerning intravenous fluids in the inpatient setting and pain medication prescribing on discharge from the hospital. Results: A total of 354 participants were enrolled and randomized, with 177 in group 1 (fluid then pain management education) and 177 in group 2 (pain management then fluid education). During the overall study period, 16 868 questions were sent to 349 learners, with 11 783 (70.0%) being opened: 10 885 (92.4%) of those opened were answered and 7175 (65.9%) of those answered were answered correctly. The differences between groups changed significantly over time, indicated by the significant interaction between educational intervention and time (P = .002). Briefly, at baseline evidence-concordant IV fluid ordered 7.2% less frequently in group 1 than group 2 (95% CI, -19.2% to 4.9%). This was reversed after training at 4% higher (95% CI, -8.2% to 16.0%) in group 1 than group 2, a more than doubling in the odds of evidence-concordant ordering (OR, 2.56, 95% CI, 0.80-8.21). Postintervention, all gains had been reversed with less frequent ordering in group 1 than group 2 (-9.5%, 95% CI, -21.6% to 2.7%). There was no measurable change in opioid prescribing behaviors at any time point. Conclusions and Relevance: In this randomized clinical trial, use of smartphone app learning modules resulted in statistically significant short-term improvement in some prescribing behaviors. However, this effect was not sustained over the long-term. Additional research is needed to understand how to sustain improvements in care delivery as a result of continuous professional development at the institutional level. Trial Registration: ClinicalTrials.gov Identifier: NCT03771482.

Imprinting on Clinical Rotations: Multisite Survey of High- and Low-Value Medical Student Behaviors and Relationship with Healthcare Intensity.

BACKGROUND: Physician behaviors are important to high-value care, and the learning environment medical students encounter on clinical clerkships may imprint their developing practice patterns. OBJECTIVES: To explore potential imprinting on clinical rotations by (a) describing high- and low-value behaviors among medical students and (b) examining relationships with regional healthcare intensity (HCI). DESIGN: Multisite cross-sectional survey PARTICIPANTS: Third- and fourth-year students at nine US medical schools MAIN MEASURES: Survey items measured high-value (n = 10) and low-value (n = 9) student behaviors. Regional HCI was measured using Dartmouth Atlas End-of-Life Chronic Illness Care data (ratio of physician visits per decedent compared with the US average, hospital care intensity index, ratio of medical specialty to primary care physician visits per decedent). Associations between regional HCI and student behaviors were examined using unadjusted and adjusted (controlling for age, sex, and year in school) logistic regression analyses, using median item ratings to summarize reported engagement in high- and low-value behaviors. KEY RESULTS: Of 2623 students invited, 1304 (50%) responded. Many reported trying to determine healthcare costs (1085/1234, 88%), but only 45% (571/1257) reported including cost details in case presentations. Students acknowledged suggesting tests solely to anticipate what their supervisor would want (1143/1220, 94%), show off their ability to generate a broad differential diagnosis (1072/1218, 88%), satisfy curiosity (958/1217, 79%), protect the team from liability (938/1215, 77%), and build clinical experience (533/1217, 44%). Students in higher intensity regions reported significantly more low-value behaviors: each one-unit increase in the ratio of physician visits per decedent increased the odds of reporting low-value behaviors by 20% (OR 1.20, 95% CI 1.04-1.38; P = 0.01). CONCLUSIONS: Third- and fourth-year medical students report engaging in both high- and low-value behaviors, which are related to regional HCI. This underscores the importance of the clinical learning environment and suggests imprinting is already underway during medical school.

Attitudes toward cost-conscious care among U.S. physicians and medical students: analysis of national cross-sectional survey data by age and stage of training.

BACKGROUND: The success of initiatives intended to increase the value of health care depends, in part, on the degree to which cost-conscious care is endorsed by current and future physicians. This study aimed to first analyze attitudes of U.S. physicians by age and then compare the attitudes of physicians and medical students. METHODS: A paper survey was mailed in mid-2012 to 3897 practicing physicians randomly selected from the American Medical Association Masterfile. An electronic survey was sent in early 2015 to all 5,992 students at 10 U.S. medical schools. Survey items measured attitudes toward cost-conscious care and perceived responsibility for reducing healthcare costs. Physician responses were first compared across age groups (30-40 years, 41-50 years, 51-60 years, and > 60 years) and then compared to student responses using Chi square tests and logistic regression analyses (controlling for sex). RESULTS: A total of 2,556 physicians (65%) and 3395 students (57%) responded. Physician attitudes generally did not differ by age, but differed significantly from those of students. Specifically, students were more likely than physicians to agree that cost to society should be important in treatment decisions (p < 0.001) and that physicians should sometimes deny beneficial but costly services (p < 0.001). Students were less likely to agree that it is unfair to ask physicians to be cost-conscious while prioritizing patient welfare (p < 0.001). Compared to physicians, students assigned more responsibility for reducing healthcare costs to hospitals and health systems (p < 0.001) and less responsibility to lawyers (p < 0.001) and patients (p < 0.001). Nearly all significant differences persisted after controlling for sex and when only the youngest physicians were compared to students. CONCLUSIONS: Physician attitudes toward cost-conscious care are similar across age groups. However, physician attitudes differ significantly from medical students, even among the youngest physicians most proximate to students in age. Medical student responses suggest they are more accepting of cost-conscious care than physicians and attribute more responsibility for reducing costs to organizations and systems rather than individuals. This may be due to the combined effects of generational differences, new medical school curricula, students' relative inexperience providing cost-conscious care within complex healthcare systems, and the rapidly evolving U.S. healthcare system.

Value-Added Activities in Medical Education: A Multisite Survey of First- and Second-Year Medical Students' Perceptions and Factors Influencing Their Potential Engagement.

PURPOSE: To describe attitudes of first- and second-year U.S. medical students toward value-added medical education, assess their self-reported desire to participate in value-added activities, and identify potentially modifiable factors influencing their engagement. METHOD: The authors conducted a cross-sectional survey of first- and second-year students at nine U.S. medical schools in 2017. Survey items measured students' attitudes toward value-added medical education (n = 7), desire to participate in value-added activities (n = 20), and factors influencing potential engagement (n = 18). RESULTS: Of 2,670 students invited to participate, 1,372 (51%) responded. Seventy-six percent (1,043/1,368) moderately or strongly agreed they should make meaningful contributions to patient care. Students' desire to participate was highest for patient care activities approximating those traditionally performed by physicians, followed by systems improvement activities and lowest for activities not typically performed by physicians. Factors increasing desire to participate included opportunities to interact with practicing physicians (1,182/1,244; 95%), patients (1,177/1,246; 95%), and residents or fellows (1,166/1,246; 94%). Factors decreasing desire to participate included making changes to the health care system (365/1,227; 30%), interacting with patients via phone or electronic communication (410/1,243; 33%), and lack of curricular time (634/1,233; 51%). CONCLUSIONS: First- and second-year medical students agree they should add value to patient care, but their desire to participate in value-added activities varies depending on the nature of the tasks. Medical schools may be able to increase students' desire to participate by enabling face-to-face interactions with patients, embedding students in health care teams, and providing dedicated curricular time.

The MD-MEd Joint-Degree Program at Vanderbilt University: Training Future Expert Medical Educators.

PROBLEM: Some medical students are drawn to medical education as an area of academic specialization. However, few options exist for medical students who wish to build a scholarly foundation for future careers in medical education. APPROACH: In 2011, Vanderbilt University School of Medicine (VUSM) and Peabody College of Education and Human Development at Vanderbilt University partnered to establish a novel dual-degree program that, through transfer of credit, allows students to graduate with both an MD and a master of education (MEd) degree in five years. The MD-MEd joint-degree program equips students with robust knowledge and skills related to general education while providing opportunities through independent studies and capstone projects to contextualize these ideas in medical education. OUTCOMES: This innovation at Vanderbilt University demonstrates the feasibility of an MD-MEd joint-degree program. MD-MEd graduates' demonstrated commitment to medical education and credentials will allow them to take on greater educational responsibilities earlier in their careers and quickly gain experience. The three author participants feel their experiences allowed them to achieve desired competencies as educators. They have each gained early experience by chairing the Student Curriculum Committee and contributing to major curricular reform at VUSM. NEXT STEPS: The authors plan to integrate specific medical education competencies into the program, which will require MD-MEd students to develop and demonstrate proficiency in the knowledge and skills expected of dedicated medical educators. Graduates' career trajectories will be tracked to explore whether they become medical educators, conduct educational research, and assume leadership positions.

How Should Resident Physicians Respond to Patients' Discomfort and Students' Moral Distress When Learning Procedures in Academic Medical Settings?

In this scenario, a medical student, Lauren, experiences moral distress because she feels that learning to perform a procedure on a patient who requested not to be used for "practice" puts her own interests above the patient's. Lauren might also worry that the resident physician is misrepresenting her abilities. The resident physician could help alleviate Lauren's distress and align her interests with the patient's by more clearly explaining the training situation to the patient and seeking the patient's approval. Lauren might also manage the situation by assuring the patient of the resident's supervisory role. This article argues that trainees should have the opportunities to practice procedures and difficult conversations in simulated settings and that institutions should support a culture of "speaking up" to ensure patients' and learners' safety.

Competency milestones for medical students: Design, implementation, and analysis at one medical school.

Competency-based assessment seeks to align measures of performance directly with desired learning outcomes based upon the needs of patients and the healthcare system. Recognizing that assessment methods profoundly influence student motivation and effort, it is critical to measure all desired aspects of performance throughout an individual's medical training. The Accreditation Council for Graduate Medical Education (ACGME) defined domains of competency for residency; the subsequent Milestones Project seeks to describe each learner's progress toward competence within each domain. Because the various clinical disciplines defined unique competencies and milestones within each domain, it is difficult for undergraduate medical education to adopt existing GME milestones language. This paper outlines the process undertaken by one medical school to design, implement and improve competency milestones for medical students. A team of assessment experts developed milestones for a set of focus competencies; these have now been monitored in medical students over two years. A unique digital dashboard enables individual, aggregate and longitudinal views of student progress by domain. Validation and continuous quality improvement cycles are based upon expert review, user feedback, and analysis of variation between students and between assessors. Experience to date indicates that milestone-based assessment has significant potential to guide the development of medical students.

Role Modeling and Regional Health Care Intensity: U.S. Medical Student Attitudes Toward and Experiences With Cost-Conscious Care.

PURPOSE: To examine medical student attitudes toward cost-conscious care and whether regional health care intensity is associated with reported exposure to physician role-modeling behaviors related to cost-conscious care. METHOD: Students at 10 U.S. medical schools were surveyed in 2015. Thirty-five items assessed attitudes toward, perceived barriers to and consequences of, and observed physician role-modeling behaviors related to cost-conscious care (using scales for cost-conscious and potentially wasteful behaviors; Cronbach alphas of 0.82 and 0.81, respectively). Regional health care intensity was measured using Dartmouth Atlas End-of-Life Chronic Illness Care data: ratio of physician visits per decedent compared with the U.S. average, ratio of specialty to primary care physician visits per decedent, and hospital care intensity index. RESULTS: Of 5,992 students invited, 3,395 (57%) responded. Ninety percent (2,640/2,932) agreed physicians have a responsibility to contain costs. However, 48% (1,1416/2,960) thought ordering a test is easier than explaining why it is unnecessary, and 58% (1,685/2,928) agreed ordering fewer tests will increase the risk of malpractice litigation. In adjusted linear regression analyses, students in higher-health-care-intensity regions reported observing significantly fewer cost-conscious role-modeling behaviors: For each one-unit increase in the three health care intensity measures, scores on the 21-point cost-conscious role-modeling scale decreased by 4.4 (SE 0.7), 3.2 (0.6), and 3.9 (0.6) points, respectively (all P < .001). CONCLUSIONS: Medical students endorse barriers to cost-conscious care and encounter conflicting role-modeling behaviors, which are related to regional health care intensity. Enhancing role modeling in the learning environment may help prepare future physicians to address health care costs.

Considering Research Outcomes as Essential Tools for Medical Education Decision Making.

As medical educators face the challenge of incorporating new content, learning methods, and assessment techniques into the curriculum, the need for rigorous medical education research to guide efficient and effective instructional planning increases. When done properly, well-designed education research can provide guidance for complex education decision making. In this Commentary, the authors consider the 2015 Research in Medical Education (RIME) research and review articles in terms of the critical areas in teaching and learning that they address. The broad categories include (1) assessment (the largest collection of RIME articles, including both feedback from learners and instructors and the reliability of learner assessment), (2) the institution's impact on the learning environment, (3) what can be learned from program evaluation, and (4) emerging issues in faculty development. While the articles in this issue are broad in scope and potential impact, the RIME committee noted few studies of sufficient rigor focusing on areas of diversity and diverse learners. Although challenging to investigate, the authors encourage continuing innovation in research focused on these important areas.

Medical students as health educators at a student-run free clinic: improving the clinical outcomes of diabetic patients.

PURPOSE: Student-run free clinics (SRFCs) provide service-learning opportunities for medical students and care to underserved patients. Few published studies, however, support that they provide high-quality care. In this study, the authors examined the clinical impact of a medical student health educator program for diabetic patients at an SRFC. METHOD: In 2012, the authors retrospectively reviewed the electronic medical records of diabetic patients who established care at Shade Tree Clinic in Nashville, Tennessee, between 2008 and 2011. They compared clinical outcomes at initial presentation to the clinic and 12 months later. They analyzed the relationship between the number of patient-student interactions (touchpoints) and change in hemoglobin A1c values between these two time points and compared the quality of care provided to best-practice benchmarks (process and outcomes measures). RESULTS: The authors studied data from 45 patients. Mean hemoglobin A1c values improved significantly from 9.6 to 7.9, after a mean of 12.5 ± 1.5 months (P < .0001). A trend emerged between increased number of touchpoints and improvement in A1c values (r = 0.06, P = .10). A high percentage of patients were screened during clinic visits, whereas a low to moderate percentage met benchmarks for A1c, LDL, and blood pressure levels. CONCLUSIONS: These findings demonstrate that a medical student health educator program at an SRFC can provide high-quality diabetes care and facilitate clinical improvement one year after enrollment, despite inherent difficulties in caring for underserved patients. Future studies should examine the educational and clinical value of care provided at SRFCs.

Role-modeling and medical error disclosure: a national survey of trainees.

PURPOSE: To measure trainees' exposure to negative and positive role-modeling for responding to medical errors and to examine the association between that exposure and trainees' attitudes and behaviors regarding error disclosure. METHOD: Between May 2011 and June 2012, 435 residents at two large academic medical centers and 1,187 medical students from seven U.S. medical schools received anonymous, electronic questionnaires. The questionnaire asked respondents about (1) experiences with errors, (2) training for responding to errors, (3) behaviors related to error disclosure, (4) exposure to role-modeling for responding to errors, and (5) attitudes regarding disclosure. Using multivariate regression, the authors analyzed whether frequency of exposure to negative and positive role-modeling independently predicted two primary outcomes: (1) attitudes regarding disclosure and (2) nontransparent behavior in response to a harmful error. RESULTS: The response rate was 55% (884/1,622). Training on how to respond to errors had the largest independent, positive effect on attitudes (standardized effect estimate, 0.32, P < .001); negative role-modeling had the largest independent, negative effect (standardized effect estimate, -0.26, P < .001). Positive role-modeling had a positive effect on attitudes (standardized effect estimate, 0.26, P < .001). Exposure to negative role-modeling was independently associated with an increased likelihood of trainees' nontransparent behavior in response to an error (OR 1.37, 95% CI 1.15-1.64; P < .001). CONCLUSIONS: Exposure to role-modeling predicts trainees' attitudes and behavior regarding the disclosure of harmful errors. Negative role models may be a significant impediment to disclosure among trainees.

International students in United States' medical schools: does the medical community know they exist?

BACKGROUND: Matriculation of international students to United States' (US) medical schools has not mirrored the remarkable influx of these students to other US institutions of higher education. METHODS: While these students' numbers are on the rise, the visibility for their unique issues remains largely ignored in the medical literature. RESULTS: These students are disadvantaged in the medical school admissions process due to financial and immigration-related concerns, and academic standards for admittance also continue to be significantly higher compared with their US-citizen peers. Furthermore, it is simply beyond the mission of many medical schools - both public and private - to support international students' education, especially since federal, state-allocated or institutional funds are limited and these institutions have a commitment to fulfill the healthcare education needs of qualified domestic candidates. In spite of these obstacles, a select group of international students do gain admission to US medical schools and, upon graduation, are credentialed equally as their US-citizen counterparts by the Accreditation Council for Graduate Medical Education (ACGME). However, owing to their foreign citizenship, these students have visa requirements for post-graduate training that may adversely impact their candidacy for residency placement. CONCLUSION: By raising such issues, this article aims to increase the awareness of considerations pertinent to this unique population of medical students. The argument is also made to support continued recruitment of international students to US medical schools in spite of these impediments. In our experience, these students are not only qualified to tackle the rigors of a US medical education, but also enrich the cultural diversity of the medical student body. Moreover, these graduates could effectively complement the efforts to augment US physician workforce diversity while contributing to healthcare disparity eradication, minority health issues, and service in medically underserved areas.