Association of a longitudinal, preclinical ultrasound curriculum with medical student performance.

INTRODUCTION: Point-of-care ultrasound (US) is used in clinical practice across many specialties. Ultrasound (US) curricula for medical students are increasingly common. Optimal timing, structure, and effect of ultrasound education during medical school remains poorly understood. This study aims to retrospectively determine the association between participation in a preclinical, longitudinal US curriculum and medical student academic performance. METHODS: All first-year medical students at a medical school in the Midwest region of the United States were offered a voluntary longitudinal US curriculum. Participants were selected by random lottery. The curriculum consisted of five three-hour hands on-sessions with matching asynchronous content covering anatomy and pathologic findings. Content was paired with organ system blocks in the standard first year curriculum at our medical school. Exam scores between the participating and non-participating students were compared to evaluate the objective impact of US education on performance in an existing curriculum. We hypothesized that there would be an association between participation in the curriculum and improved medical student performance. Secondary outcomes included shelf exam scores for the surgery, internal medicine, neurology clerkships and USMLE Step 1. A multivariable linear regression model was used to evaluate the association of US curriculum participation with student performance. Scores were adjusted for age, gender, MCAT percentile, and science or engineering degree. RESULTS: 76 of 178 students applied to participate in the curriculum, of which 51 were accepted. US curriculum students were compared to non-participating students (n = 127) from the same class. The US curriculum students performed better in cardiovascular anatomy (mean score 92.1 vs. 88.7, p = 0.048 after adjustment for multiple comparisons). There were no significant differences in cumulative cardiovascular exam scores, or in anatomy and cumulative exam scores for the gastroenterology and neurology blocks. The effect of US curriculum participation on cardiovascular anatomy scores was estimated to be an improvement of 3.48 points (95% CI 0.78-6.18). No significant differences were observed for USMLE Step 1 or clerkship shelf exams. There were no significant differences in either preclinical, clerkship or Step 1 score for the 25 students who applied and were not accepted and the 102 who did not apply. CONCLUSIONS: Participation in a preclinical longitudinal US curriculum was associated with improved exam performance in cardiovascular anatomy but not examination of other cardiovascular system concepts. Neither anatomy or comprehensive exam scores for neurology and gastrointestinal organ system blocks were improved.

Incorporating active learning into a psychiatry clerkship: does it make a difference?

OBJECTIVE: Medical students' satisfaction with the psychiatry clerkship, sense of preparedness for an institutional Objective Structured Clinical Exam (OSCE), expressed likelihood of choosing psychiatry as a specialty, and National Board of Medical Examiners (NBME) psychiatry shelf-examination scores were compared after a curriculum based on Active Learning (AL) techniques was introduced. METHOD: In consecutive academic years, two groups of students were compared after completing a 1-month psychiatry clerkship. The first group (N=108) received traditional lectures, and the second (N=102) was taught via AL. Participants were surveyed regarding satisfaction, sense of preparedness for an institutional OSCE, and expressed likelihood of choosing psychiatry as a specialty. NBME psychiatry shelf-examination scores were analyzed; independent-samples t-tests were used to evaluate the data. RESULTS: Satisfaction and sense of preparedness for the institutional OSCE increased with AL techniques. NBME scores were not significantly different between groups. Professed likelihood of choosing psychiatry as a specialty did not increase with the interventional curriculum. CONCLUSION: We confirmed findings from previous studies that student satisfaction improves with active learning (AL). Sense of preparedness for the OSCE examination improved with AL, as well. NBME psychiatry exam scores and professed interest in psychiatry as a specialty were not different from those taught in a traditional format.

Incoming resident experience and comfort with procedures designated as "basic".

BACKGROUND: Recent consensus guidelines have recommended uniformity in procedural training in family medicine residency programs. The consensus documentation suggestions are based on expectations of procedural training prior to residency. Few studies have evaluated the perceived baseline level of procedural competency prior to residency training. METHODS: Twenty incoming PGY-1 residents completed a "procedural experience survey," asking respondents to identify their prior experience and current comfort levels with 19 "basic" procedures. RESULTS: For seven procedures, no prior experience was the most common response. For 15 of the 19 procedures, at least one respondent reported no prior experience. The residents' self-reported comfort levels varied widely for each procedure. CONCLUSIONS: The survey results showed that experience and comfort with procedures often performed in family medicine varies widely among incoming residents. This raises the question of whether documentation of resident competence in these procedures should be recommended as a baseline and monitored throughout residency training.

Interrater and intrarater reliability of the active hip abduction test.

STUDY DESIGN: Clinical measurement. OBJECTIVES: To determine the interrater and intrarater reliability of the active hip abduction (AHAbd) test. BACKGROUND: The AHAbd test is used to assess lumbopelvic movement during a dynamic lower limb activity. The test has previously been shown to predict low back pain development during a prolonged standing exposure in previously asymptomatic individuals. As an observation-based assessment for which rater reliability has not been established, similar scoring on the test between clinicians is essential. METHODS: One hundred twenty-eight video clips of participants performing the AHAbd test were recorded. Sixteen practicing physical therapists scored test performance by viewing 20 preselected videos to establish interrater reliability. Fourteen of the 16 raters rescored the videos after a 3-week period to establish intrarater reliability. Demographic data were collected for all raters. Intraclass correlation coefficients (ICCs) were calculated for reliability statistics. Correlations were performed between demographic data and ICCs. RESULTS: Interrater reliability (ICC2,1) for the test using the 4-point scale was 0.70 (95% confidence interval [CI]: 0.56, 0.84) and 0.59 (95% CI: 0.43, 0.76) when the scale was dichotomized into positive/negative scores. Intrarater reliability (ICC3,1) was 0.74 on average. Demographic characteristics were not significantly associated with reliability scores. CONCLUSION: Interrater and intrarater reliability for scoring of the AHAbd test by practicing clinicians was similar, regardless of experience level or practice setting. The AHAbd test can be considered to be a reliable observational tool.

Adding new tools to the black bag--introduction of ultrasound into the physical diagnosis course.

INTRODUCTION: Ultrasound, a versatile diagnostic modality that permits real-time visualization at the patient's bedside, can be used as an adjunct in teaching physical diagnosis (PD). AIMS: (1) to study the feasibility of incorporating ultrasound into PD courses and (2) determine whether learners can demonstrate image recognition and acquisition skills. PROGRAM DESCRIPTION: Three hundred seven second-year medical students were introduced to cardiovascular and abdominal ultrasound scanning after training in the physical examination. This consisted of a demonstration of the ultrasound examination, followed by practice on standardized patients (SPs). Pre-post tests were administered to evaluate students' knowledge and understanding of ultrasound. Students performed an ultrasound examination during the PD final examination. PROGRAM EVALUATION: Pre-post test data revealed significant improvements in image recognition. On the final exam, the highest scores (98.4%) were obtained for the internal jugular vein and lowest scores (74.6%) on the Focused Assessment with Sonography for Trauma images. Eighty-nine percent of students' surveyed felt ultrasound was a valuable tool for physicians. DISCUSSION: An introductory ultrasound course is effective in improving medical students' acquisition and recognition of basic cardiovascular and abdominal ultrasound images. This innovative program demonstrates the feasibility of incorporating portable ultrasound as a learning tool during medical school.

Strategies for identifying students at risk for USMLE step 1 failure.

BACKGROUND AND OBJECTIVES: Failing Step 1 of the US Medical Licensing Examination (USMLE) or a delay in taking the exam can negatively affect a medical student's ability to match into a residency program. Unfortunately, identifying students at risk for failing Step 1 is challenging, but it is necessary to provide proactive educational support. The purpose of this study was to develop a strategy to identify students at risk for failing Step 1. METHODS: Using a retrospective study design, 256 students from the class of 2008 were eligible for the study. Independent variables included Medical College Admission Test (MCAT) scores and cumulative grades from years 1--2 of medical school. The dependent variable was their score on the USMLE Step 1. Variables with a significant univariate relationship were loaded into a series of binary logistic regression models. A receiver operating characteristic (ROC) curve examined the significant variables. RESULTS: Both year-2 standard score and the MCAT biological sciences score were significant as predictors of failure. The ROC curve provided a range of values for establishing a cutoff value for each significant variable. CONCLUSION: Using internal and external predictors, it is possible to identify students at risk for failing Step 1 of the USMLE.

Incorporating Geographic Information Systems (GIS) into program evaluation: lessons from a rural medicine initiative.

PURPOSE: To address the shortage of physicians practicing in rural areas of Michigan, the Wayne State University School of Medicine developed an integrated rural core curriculum to interest students in rural practice careers. Here we focus on the evaluation strategy used to determine the extent to which students in the new rural medicine interest group who self-identified as selecting a rural clerkship or externship did secure a clinical training experience in a rural setting. METHODS: Three measures of rurality were compared to determine whether students were placed in rural training settings: (1) the percentage of the county living in rural areas; (2) a county-level dichotomous measure of rural/nonrural; and (3) a dichotomous measure based on urban area boundaries within the county. Practice address and geographic data were integrated into geographic information systems software, which we used to map out rural characteristics of Michigan counties through a process called thematic mapping; this shows characteristic variation by color-shading geographic features. In addition, reference maps were created showing the boundaries of urban areas and metropolitan/micropolitan areas. Once these processes were completed, we overlaid the practice location on the contextual-level geographic features to produce a visual representation of the relationship between student placement and rural areas throughout the state. RESULTS: The outcome of student placement in rural practices varied by the definition of rural. We concluded that, although students were not placed in the most rural areas of Michigan, they received clerkship or externship training near rural areas or in semirural areas. CONCLUSION: This process evaluation had a direct impact on program management by highlighting gaps in preceptor recruitment. A greater effort is being made to recruit physicians for more rural areas of the state rather than urban and semirural areas. Geographic information systems mapping also defined levels of ruralism for students to help them make informed selections of training sties. This is especially important for students who are not sure about a rural experience and might be discouraged by placement in a remote rural area.

Qualitative and quantitative outcomes of audience response systems as an educational tool in a plastic surgery residency program.

BACKGROUND: In-training evaluations in graduate medical education have typically been challenging. Although the majority of standardized examination delivery methods have become computer-based, in-training examinations generally remain pencil-paper-based, if they are performed at all. Audience response systems present a novel way to stimulate and evaluate the resident-learner. The purpose of this study was to assess the outcomes of audience response systems testing as compared with traditional testing in a plastic surgery residency program. METHODS: A prospective 1-year pilot study of 10 plastic surgery residents was performed using audience response systems-delivered testing for the first half of the academic year and traditional pencil-paper testing for the second half. Examination content was based on monthly "Core Quest" curriculum conferences. Quantitative outcome measures included comparison of pretest and posttest and cumulative test scores of both formats. Qualitative outcomes from the individual participants were obtained by questionnaire. RESULTS: When using the audience response systems format, pretest and posttest mean scores were 67.5 and 82.5 percent, respectively; using traditional pencil-paper format, scores were 56.5 percent and 79.5 percent. A comparison of the cumulative mean audience response systems score (85.0 percent) and traditional pencil-paper score (75.0 percent) revealed statistically significantly higher scores with audience response systems (p = 0.01). Qualitative outcomes revealed increased conference enthusiasm, greater enjoyment of testing, and no user difficulties with the audience response systems technology. CONCLUSIONS: The audience response systems modality of in-training evaluation captures participant interest and reinforces material more effectively than traditional pencil-paper testing does. The advantages include a more interactive learning environment, stimulation of class participation, immediate feedback to residents, and immediate tabulation of results for the educator. Disadvantages include start-up costs and lead-time preparation.

Retention of medical genetics knowledge and skills by medical students.

PURPOSE: To determine whether specific knowledge and skills medical students acquire after completing a Year 1 genetics course are retained at the end of Year 3. METHODS: A genetics case was developed for an observed structured clinical exam at the end of Year 3. The case involved a pregnant patient who underwent population screening for cystic fibrosis and is identified as a carrier of a common mutation. Student's performance in completing eight essential genetic tasks taught in Year 1 was assessed by their ability to apply these concepts in the Year 3 observed structured clinical exam. RESULTS: A total of 212 students were included in the study. Performance on the essential tasks revealed that students were better able to discuss inheritance pattern (73.1%). Students were less likely to calculate and discuss fetal risk (25%), discuss the option of prenatal diagnosis if the father is a carrier (25%), and ask about a family history of cystic fibrosis (36.8%). Only half (50%) explained the test result and implications to the patient. There was no correlation between individual student exam scores in Year 1 and the eight essential genetics tasks scores assessed in the observed structured clinical exam (r = 0.003, P < or = 0.67). CONCLUSION: Third year medical students do not retain medical genetics knowledge and skills learned in the first year of medical school. Medical schools need to integrate genetics curriculum through the continuum of the 4 years of medical school.

The effectiveness of streaming video on medical student learning: a case study.

Information technology helps meet today's medical students' needs by providing multiple curriculum delivery methods. Video streaming is an e-learning technology that uses the Internet to deliver curriculum while giving the student control of the content's delivery. There have been few studies conducted on the effectiveness of streaming video in medical schools. A 5-year retrospective study was conducted using three groups of students (n = 1736) to determine if the availability of streaming video in Years 1-2 of the basic science curriculum affected overall Step 1 scores for first-time test-takers. The results demonstrated a positive effect on program outcomes as streaming video became more readily available to students. Based on these findings, streaming video technology seems to be a viable tool to complement in-class delivery methods, to accommodate the needs of medical students, and to provide options for meeting the challenges of delivering the undergraduate medical curriculum. Further studies need to be conducted to continue validating the effectiveness of streaming video technology.

A pilot study of comprehensive ultrasound education at the Wayne State University School of Medicine: a pioneer year review.

OBJECTIVE: Ultrasound is a versatile diagnostic modality used in a variety of medical fields. Wayne State University School of Medicine (WSUSOM) is one of the first medical schools in the United States to integrate an ultrasound curriculum through both basic science courses and clinical clerkships. METHODS: In 2006, 25 portable ultrasound units were donated to WSUSOM. First-year medical students were provided an ultrasound curriculum consisting of 6 organ-system sessions that addressed the basics of ultrasound techniques, anatomy, and procedural skills. After the last session, students were administered 2 anonymous and voluntary evaluations. The first assessed their overall experience with the ultrasound curriculum, and the second assessed their technical skills in applying ultrasound techniques. RESULTS: Eighty-three percent of students agreed or strongly agreed that their experience with ultrasound education was positive. On the summative evaluation, nearly 91% of students agreed or strongly agreed that they would benefit from continued ultrasound education throughout their 4 years of medical school. Student performance on the technical assessment was also very positive, with mean class performance of 87%. CONCLUSIONS: As residency programs adopt ultrasound training, medical school faculty should consider incorporating ultrasound education into their curriculum. Portable ultrasound has the potential to be used in many different settings, including rural practice sites and sporting events. The WSUSOM committee's pilot ultrasound curriculum will continue to use student feedback to enhance the ultrasound experience, helping students prepare for challenges that they will face in the future.

Evaluating the phobias, attitudes, and cultural competence of Master of Social Work students toward the LGBT populations.

BACKGROUND: Evidence suggests there is bias toward lesbian, gay, bisexual, and transgender (LGBT) persons by social workers; unfortunately, little research has been conducted to examine Master of Social Work (MSW) students' views toward these populations. The purpose of this study was to develop an assessment scale to evaluate the attitudes, phobias, and cultural competence of MSW students toward the LGBT populations. METHODS: An assessment scale was developed and administered to MSW students (n = 173) at a Midwestern American university. RESULTS: The majority of MSW students reported low phobia and a positive attitude toward the LGBT populations, yet participants reported having a low level of cultural competence in serving LGBT clients. CONCLUSION: More education and training is needed for MSW students to effectively serve the LGBT populations.

Wireless Handheld Computers in the Undergraduate Medical Curriculum.

Wayne State University Medical School has implemented wireless handheld computers or PocketPCs (PPCs) into all four years of the undergraduate curriculum. A transition from a passive to an interactive learning environment in the classroom, as well as administrative solutions for monitoring patient encounter data by students in their clinical rotations was fostered by this educational technology. Implementation of the wireless devices into the curriculum will be described in this report. This will include the technical specifications and justification for the required device, as well as a detailed discussion of the different applications used for educational and administrative purposes by the preclinical and clinical students. Outcomes from the educational and administrative aspects of the project will also be presented in this report.

Promoting informed decision making: evaluating a community-based prostate health awareness program.

BACKGROUND: For health educators, the controversy surrounding routine prostate cancer screening provides curriculum development and delivery challenges. The purpose of this study was to evaluate a community-based prostate health awareness program. METHODS: Using a pretest-posttest design, participants were recruited from community-based organizations to assess the effectiveness of the program in the areas of knowledge gain, short-term intentions, and changing prostate health behaviors. RESULTS: Many of the participants reported having been tested for prostate cancer, yet there was a lower than expected pretest knowledge base. There were significant increases in knowledge on the posttest and some impact on short-term intentions and behavior. DISCUSSION: A community-based prostate health awareness program prior to the patient-physician encounter can assist health-care professionals in the education process and give men the tools to make an informed decision.

UME-21 local evaluation initiatives: contributions and challenges.

BACKGROUND AND OBJECTIVES: The 18 medical schools involved in the UME-21 initiative developed innovative curricula and evaluation strategies. While there was significant variation in how schools approached the evaluation process, there were common methodological issues and challenges affecting the reliability of scores and validity of interpretations regarding outcomes. This paper explores these issues and challenges, using experiences from selected UME-21 schools. METHODS: Four evaluation issues and strategies are discussed: instrument development, study design, process evaluation using formative evaluation methods, and qualitative strategies. Within each discussion, examples from a UME-21 school are presented. RESULTS: The four evaluation strategies offered the flexibility to match local evaluation needs with an effective approach to evaluations. CONCLUSIONS/IMPLICATIONS: The school-level evaluation requirements by the UME-21 initiative provided schools the flexibility to design individualized evaluation strategies, yet also encouraged collaboration among evaluators. While this strategy resulted in many successes at the school level, it also served to identify common methodological challenges that can be used as a guide for other schools in implementing and evaluating curricula.

Measurement practices: methods for developing content-valid student examinations.

Measurement experts generally agree that a systematic approach to test construction will probably result in an instrument with sound psychometric properties. One fundamental method is called the blueprint approach to test construction. A test blueprint is a tool used in the process for generating content-valid exams by linking the subject matter delivered during instruction and the items appearing on the test. Unfortunately, this procedure as well as other educational measurement practices is often overlooked A survey of curriculum administrators at 144 United States and international medical schools was conducted to assess the importance and prevalence of test blueprinting in their school. Although most found test blueprinting to be very important, few require the practice. The purpose of this paper is to review the fundamental principals associated with achieving a high level of content validity when developing tests for students. The short-term efforts necessary to develop and integrate measurement theory into practice will lead to long-term gains for students, faculty and academic institutions.

Evaluation of a preparatory community-based prostate health education program.

BACKGROUND: Medical organizations are divided on the issue of screening for prostate cancer, yet there is agreement that men should be educated about prostate health. Shared decision making involves patients and practitioners in this process, yet some men need preparatory education prior to the physician encounter. This study assessed the effectiveness of a community prostate health awareness program, focusing on men at risk for prostate cancer. METHODS: Participants were given a pretest and a posttest to assess knowledge gains and impact on short-term intentions to address their prostate health. RESULTS: There was a statistically significant increase in knowledge. Short-term intentions increased for those participants meeting the inclusion criteria. DISCUSSION: Community outreach programs remain an excellent vehicle to educate the public and complement the efforts of health care providers.

An Integrated Approach for Evaluating Students' Achievement of Clinical Objectives.

During the clinical phase of undergraduate medical education (UME) students are often geographically dispersed and assigned to preceptors throughout the community. Monitoring, documenting, and evaluating their clinical experiences and achievement of clinical objectives in this venue becomes a challenge, especially for large UME programs. The purpose of this manuscript is to discuss a method for developing and implementing a school-wide evaluation system for the clinical phase of UME. This type of evaluation system links students' clinical experiential data with the objectives of a clerkship, using technological advances, such as the Personal Digital Assistant (PDA), Internet, and intranet. Clerkship directors are provided realtime reports on student's progress toward achieving clerkship objectives and are able to monitor the clinical activities of the clerkship. Students on the other hand, will be empowered to take more control of their educational experiences by monitoring their own progress.