Factors Associated with Preference for Vaginal Birth among Undergraduate Students from Southern Brazil.

BACKROUND: Cesarean rates in Brazil have reached over 50 percent of all births. Multiple factors have been studied aiming to explain these rates. Mode of delivery preferences among university students may provide insights into strategies to reduce those numbers. METHODS: A cross-sectional study was conducted enrolling students who attended Health Sciences and Human Sciences undergraduate programs in 2013. Participants answered a semi-structured questionnaire about which mode of delivery they considered the "best" (less risky and more beneficial) and the "preferred" mode (the one they would choose for themselves or their partners). Pearson chi-square was used to assess association among variables. Multiple regression analysis identified independent factors associated with the outcome measures. RESULTS: Among the 797 students who provided complete responses (76% response rate), the mean age was 22.6 years, 61.6 percent were female, 2.6 percent had previous pregnancies, and 56.7 percent were born by cesarean. Vaginal birth was chosen as the "best" mode of delivery by 91.2 percent, and the "preferred" mode by 75.5 percent of students. Being male, born vaginally, and a Health Sciences student was associated with a preference for vaginal birth. Being a Health Sciences student and born vaginally was associated with considering vaginal birth the "best" mode of delivery. CONCLUSIONS: Our findings indicate that the recognition of the benefits of vaginal birth do not always translate into a personal preference for vaginal birth. The student's own mode of birth was a strong predictor of their preferences for mode of delivery.

How to set standards on performance-based examinations: AMEE Guide No. 85.

This AMEE Guide offers an overview of methods used in determining passing scores for performance-based assessments. A consideration of various assessment purposes will provide context for discussion of standard setting methods, followed by a description of different types of standards that are typically set in health professions education. A step-by-step guide to the standard setting process will be presented. The Guide includes detailed explanations and examples of standard setting methods, and each section presents examples of research done using the method with performance-based assessments in health professions education. It is intended for use by those who are responsible for determining passing scores on tests and need a resource explaining methods for setting passing scores. The Guide contains a discussion of reasons for assessment, defines standards, and presents standard setting methods that have been researched with performance-based tests. The first section of the Guide addresses types of standards that are set. The next section provides guidance on preparing for a standard setting study. The following sections include conducting the meeting, selecting a method, implementing the passing score, and maintaining the standard. The Guide will support efforts to determine passing scores that are based on research, matched to the assessment purpose, and reproducible.

Examining changes in certification/licensure requirements and the international medical graduate examinee pool.

Changes in certification requirements and examinee characteristics are likely to influence the validity of the evidence associated with interpretations made based on test data. We examined whether changes in Educational Commission for Foreign Medical Graduates (ECFMG) certification requirements over time were associated with changes in internal medicine (IM) residency program director ratings and certification examination scores. Comparisons were made between physicians who were ECFMG-certified before and after the Clinical Skills Assessment (CSA) requirement. A multivariate analysis of covariance was conducted to examine the differences in program director ratings based on CSA cohort and whether the examinees emigrated for undergraduate medical education (national vs. international students). A univariate analysis of covariance was conducted to examine differences in scores from the American Board of Internal Medicine (ABIM) Internal Medicine Certification Examination. For both analyses, United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores were used as covariates. Results indicate that, of those certified by ECFMG between 1993 and 1997, 17 % (n = 1,775) left their country of citizenship for undergraduate medical education. In contrast, 38 % (n = 1,874) of those certified between 1999 and 2003 were international students. After adjustment by covariates, the main effect of cohort membership on the program director ratings was statistically significant (Wilks' λ = 0.99, F 5, 15391 = 19.9, P < 0.001). However, the strength of the relationship between cohort group and the ratings was weak (η = 0.01). The main effect of migration status was statistically significant and weak (Wilks' λ = 0.98, F 5,15391 = 45.3, P < 0.01; η = 0.02). Differences in ABIM Internal Medicine Certification Examination scores based on whether or not CSA were required was statistically significant, although the magnitude of the association between these variables was very small. The findings suggest that the implementation of an additional evaluation of skills (e.g., history-taking, physical examination) as a prerequisite to postgraduate medical education (residency) provides some additional, relevant data to those who select ECFMG-certified residents.

Effect of first-encounter pretest on pass/fail rates of a clinical skills medical licensure examination.

Due to the high-stakes nature of medical exams it is prudent for test agencies to critically evaluate test data and control for potential threats to validity. For the typical multiple station performance assessments used in medicine, it may take time for examinees to become comfortable with the test format and administrative protocol. Since each examinee in the rotational sequence starts with a different task (e.g., simulated clinical encounter), those who are administered non-scored pretest material on their first station may have an advantage compared to those who are not. The purpose of this study is to investigate whether pass/fail rates are different across the sequence of pretest encounters administered during the testing day. First-time takers were grouped by the sequential order in which they were administered the pretest encounter. No statistically significant difference in fail rates was found between examinees who started with the pretest encounter and those who encountered the pretest encounter later in the sequence. Results indicate that current examination administration protocols do not present a threat to the validity of test score interpretations.

U.S. citizens who obtain their medical degrees abroad: an overview, 1992-2006.

The number of U.S. citizens attending medical schools outside the United States and Canada has increased recently. Because these people tend to return to the United States to practice medicine, it is important to know more about their characteristics and educational experiences. Based on summary data from certifying examinations, U.S. citizens trained abroad do not perform as well as either other international medical graduates (IMGs) or U.S. graduates. Moreover, they are more likely than non-U.S. citizens to be engaged in primary care activities. Changes in the composition of the IMG pool could affect the makeup and quality of the U.S. physician workforce.

International exchange programs and U.S. medical schools.

BACKGROUND: International health experiences have been shown to increase students' and residents' interest in public health, their likelihood of choosing a career in primary care, and their commitment to serving the underserved. The purpose of the current study is to describe the international experiences available to students and residents at U.S. allopathic medical schools. METHOD: An online survey was conducted to collect information about the types of international opportunities in medical education provided to faculty, students, and residents at U.S. allopathic medical schools. RESULTS: Reponses from 103 representatives of 96 U.S. allopathic medical schools were included in the analysis. A variety of opportunities for students and residents was reported, with 59% of the respondents reporting elective rotations for residents, 11% reporting a global health track for students, and 45% reporting opportunities to perform preclinical research abroad. CONCLUSIONS: Despite associated costs and risks, U.S. medical schools are developing and refining international health experiences for medical students and residents.

Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam.

BACKGROUND: The ability to communicate effectively with patients is an essential element of a physician's clinical expertise. METHOD: As part of the USMLE Step 2 Clinical Skills exam, standardized patients (SPs) provided ratings of communication and interpersonal skills (CIS) along three dimensions. Assessment data from a one-year (2006) cohort of graduates of international medical schools (IMGs) were analyzed and psychometric characteristics of the CIS measures are described. RESULTS: Female and native English-speaking physicians received slightly higher CIS ratings than did males and examinees of other language backgrounds across all dimensions. However, no significant examinee by SP gender interaction effect was detected. Correlations with other measures support the construct validity of the CIS scale. CONCLUSIONS: Results of this preliminary study indicate that properly trained SPs can provide accurate and defensible ratings of IMGs' communication and interpersonal skills.

Migration of doctors for undergraduate medical education.

BACKGROUND: Global shortages of healthcare workers in both developed and developing countries are of great concern. Research on physician migration typically focuses on medical school graduates, most often those seeking postgraduate training opportunities elsewhere. DESCRIPTION: An overview of medical school migration patterns is presented in this paper. To put this phenomenon into the broader context of global physician migration, data is also presented on the distribution of medical schools, physician density, the flow of international medical graduates to the US, and the present composition of the US physician workforce. RESULTS: Results of the study indicate that many individuals leave their home country for undergraduate medical education. CONCLUSIONS: Given the movement of students and physicians, both for medical school and for advanced training opportunities, it is evident that some medical schools in the world are training doctors for their home country as well as for the international labor market. Overall, given the internationalization of medical education, collaborative efforts will be needed to develop an adequate, balanced, and well-trained global physician workforce.

Does composition medium affect the psychometric properties of scores on an exercise designed to assess written medical communication skills?

The ECFMG Clinical Skills Assessment (CSA) was developed to evaluate whether graduates of international medical schools are ready to enter graduate training programs in the United States. The performance-based patient note exercise is specifically used to assess an examinee's ability to summarize, synthesize and interpret the data collected in a patient interview. The purpose of this study was to investigate whether choice of composition method (typing or writing) affected the psychometric properties of the scores. Using data for a 1-year period, the validity and reliability of typed and written notes was contrasted. Although the characteristics of individuals who chose whether or not to type were different, the statistical analyses indicated that, controlling for examinee ability, physician examiner ratings of the written summaries were not influenced by composition method. The results of this study suggest that the psychometric properties of the patient note scores are invariant with respect to composition method.

Educational commission for foreign medical graduates certification and specialty board certification among graduates of the Caribbean medical schools.

BACKGROUND: To determine the variability among Caribbean medical schools in the performance of their students against markers of quality. METHOD: The Educational Commission for Foreign Medical Graduates (ECFMG) database and the American Medical Association (AMA) Masterfile were linked. Analyses were restricted to ECFMG applicants, from 1980 to 2000, who attended the 12 Caribbean medical schools that produced 200 or more U.S. International Medical School Graduates (USIMGs). RESULTS: The schools produced 17,049 USIMG (48% of all USIMGs) and 5,840 non-USIMG (5% of all non-USIMGs) applicants for ECFMG certification. By school, the ECFMG certification rate ranged from 28% to 86% for USIMGs and from 27% to 82% for non-USIMGs. Specialty Board certification for graduates of these schools also varied widely. CONCLUSIONS: Medical schools in the Caribbean play an increasingly important role in supplying the U.S. workforce. There is considerable variability among them, both in the numbers of physicians they produce and the performance of their students against indicators of quality such as ECFMG and specialty Board certification.

A comparison of the characteristics and examination performances of U.S. and non-U.S. citizen international medical graduates who sought Educational Commission for Foreign Medical Graduates certification: 1995-2004.

BACKGROUND: International medical graduates (IMGs) make up a substantial part of the U.S. physician workforce. Unfortunately, little is known about the demographic and performance characteristics of these doctors, especially over time. METHOD: Educational Commission for Foreign Medical Graduates (ECFMG) applicants from 1995-2004 were studied. The characteristics and examination performances of U.S. citizen and non-U.S. citizen IMGs were contrasted. RESULTS: ECFMG applications from U.S. citizen IMGs have been increasing. Compared to non-U.S. citizen IMGs, these individuals perform less well on some of the certification examinations, especially in the last 5 years, but are more likely to eventually achieve certification. Based on those medical students/graduates who took the examinations, the performance of U.S. citizen IMGs varied considerably from one medical school to another. CONCLUSION: The composition of the pool of ECFMG-certified IMGs available for graduate medical education has changed over the past 10 years. Shifts in the characteristics and abilities of IMGs are likely to have some future impact on health care delivery in the United States.

Effects of case characteristics on encounter time in a high-stakes standardized patient examination.

BACKGROUND: Data from national surveys indicate that patient characteristics could influence the time spent by physicians interviewing and assessing patients. The purposes of this investigation were to gather information regarding the relationship between encounter time and case characteristics for simulated clinical encounters and to provide evidence that the time provided to gather data was adequate. Timing data was extracted from United States Medical Licensing Examination Step 2 Clinical Skills. METHOD: To test the relative effects of case characteristics on encounter time, an analysis of variance was conducted with encounter time as the dependent variable and case characteristics as the independent variables. RESULTS: Mean encounter times were computed based on the case characteristics. Station format (history only, history and physical examination, telephone cases) predicted the most variance in encounter time (16%). CONCLUSIONS: The extent to which examination content is balanced from administration to administration ensures a mix of cases that provides adequate time limits for examinees.

The medical education of United States citizens who train abroad.

BACKGROUND: Graduates of international medical schools (IMGs) make up approximately one quarter of the physician workforce in the United States. Among IMGs are a number of US citizens (USIMGs) who take graduate training positions and ultimately practice in the United States. Compared with graduates of US medical schools (USMGs), relatively little is known about the undergraduate educational experiences of these US citizens. The objective of this study was to identify the schools that produce the most USIMGs and to describe the educational experiences and examination performance of graduates of these schools. METHODS: The 10 largest schools were identified based on the number of USIMGs who were certified by the Educational Commission for Foreign Medical Graduates between 2001 and 2004. Information about the institutions was collected from the International Medical Education Directory; educational experience information was gathered from 100 randomly selected transcripts and from a survey that was completed by 418 graduates of the schools. These data were compared with information about USMGs from the curriculum management and information tool and the graduation questionnaire of the Association of American Medical Colleges. Performance on steps 1 and 2CK of the United States Medical Licensing Examination were also analyzed. RESULTS: Some differences existed between USIMGs and USMGs in the required clinical clerkships, the sites where educational experiences occurred, and the special topics that were covered. USMGs and non-US citizen IMGs had better examination scores than USIMGs. CONCLUSION: There are many similarities; however, there are some noteworthy differences between the educational experiences of USIMGs and USMGs. Further work is needed to better understand the educational experiences of USIMGs, particularly in the clinical clerkships.

Using factor analysis to evaluate checklist items.

BACKGROUND: Relatively little research has been published examining the use of factor analytic techniques to improve the psychometric qualities of performance assessments. The purpose of the current investigation is to illustrate the potential use of structural equation modeling (SEM) with scores from a standardized patient examination. METHOD: Checklist items for a single case were examined to compare two models within a SEM framework: a single, underlying "data gathering" construct and a two-factor model consisting of medical history interviewing and physical examination maneuvers. RESULTS: The chi2 statistics obtained were statistically significant for both the single factor (chi2=1645.8, p<.001) and the two-factor model, (chi2=1373.1; p<.001), indicating marginal model-data fit. A difference statistic was calculated (chi2=320.9; p<.001), suggesting that the two-factor model has better fit. CONCLUSIONS: The parameters estimated by each model could be used to evaluate checklist item performance. The results of this investigation illustrate the utility of this method in the analysis of checklist item data.

Specialty board certification among U.S. citizen and non-U.S. citizen graduates of international medical schools.

BACKGROUND: Graduates of international medical schools (IMGs) make up approximately one-quarter of the physician population and play a key role in the provision of health care in the United States. This study investigated whether they differ from U.S. medical graduates (USMGs) in specialty board certification. METHOD: The study compared USMGs, U.S. citizen IMGs (USIMGs), and non-U.S. citizen IMGs (non-USIMGs) who graduated from medical school between 1958 and 1994 and were involved in direct patient care in 2003. RESULTS: There is variability among the specialties, but overall USMGs have the highest specialty certification rates followed by non-USIMGs, and USIMGs. Among recent medical school graduates, non-USIMGs have certification rates that are comparable to USMGs. CONCLUSIONS: IMGs have lower board-certification rates than USMGs, although a sizeable majority has achieved board certification in the specialty they practice. There are differences between non-USIMGs and USIMGs, with the former more likely to become board certified.

Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA).

Throughout the 40 year history of standardized patient assessments and OSCEs, there have been numerous advancements, including many that involve scoring the simulated clinical encounters. While there is no clear agreement on how examinees' performance should be documented or scored in an encounter, there is a consensus that several well-chosen SP encounters are required to produce reliable examinee scores. There also continues to be some debate as to who should do the scoring on an SP-based assessment. While logistics and cost will certainly play a role, it is probably best to use the person who is most familiar with the domain being assessed. In some instances this will be the SP; in others, an outside observer or content expert. Finally, with the growing use of OSCEs for summative purposes (e.g. certification, licensure), special attention must be paid to fairness issues. Since the same test form cannot be used day after day, examinee scores must be 'equated', taking into account the psychometric properties of scores from individual cases and individual SPs. To date, the CSA has been one of the highest-volume, high-stakes, standardized patient assessments to be developed and successfully administered. In 2003 alone, over 11 500 IMGs were tested. The early conceptual framework for this assessment was synthesized from the research endeavours of several notable individuals, including, amongst many others, Harden et al. 1975, Swanson & Stillman, 1990, Newble & Swanson, 1988, Vu et al. 1992 and Colliver, 1995. The early prototype administrations of the CSA, including many operational research studies, were supported and guided by Dr Friedman Ben-David, Friedman et al. 1991, 1993, Stillman et al. 1992, and Sutnick et al. 1993, 1995.

Investigating gender-related construct-irrelevant components of scores on the written assessment exercise of a high-stakes certification assessment.

The ECFMG Clinical Skills Assessment (CSA) was developed to evaluate whether graduates of international medical schools (IMGs) are ready to enter graduate training programs in the United States. The patient note (PN) exercise is specifically used to assess a candidate's ability to summarize and synthesize the data collected in a simulated patient interview. In a 1-year period, over 7700 first time takers completed the CSA, resulting in over 77,000 physician-based performance ratings. An initial pilot study indicated that, based solely on handwriting, the raters were able to correctly classify the gender of the candidate approximately 70% of the time. This result, combined with the fact that the notes are holistically scored, suggests that rating bias is possible. The purpose of this study was to investigate whether the gender of the candidate, the gender of the performing standardized patient, and the gender of the rater had any impact on scores. An analysis of covariance (ANCOVA) indicated that there was no significant interaction between candidate and rater gender. Female candidates significantly outperformed males, regardless of rater gender (p < 0.01, effect size = 0.23). The results of this study suggest that, based on rater, SP, and candidate characteristics, the validity of the PN ratings is not compromised.

The influence of ethnicity on patient satisfaction in a standardized patient assessment.

PURPOSE: To study possible differences in patient satisfaction ratings as a function of physician and patient ethnicity in a standardized patient (SP) performance-based assessment. METHOD: Data from 334,397 ECFMG Clinical Skills Assessment (CSA) simulated clinical encounters were analyzed. A between-groups analysis of covariance was conducted, with independent variables consisting of SP and candidate ethnicity. RESULTS: Although a significant interaction between SP and candidate ethnicity was found, averaged over all encounters the SPs were equally satisfied with doctors from all cohorts. In general, SPs provided higher satisfaction ratings for racially concordant pairings. Among the ethnic groups of SPs, Asians were the least satisfied. CONCLUSIONS: Satisfaction can vary as a function of the ethnicity of the SP. These results are similar to studies of actual patient encounters that show Asians as less satisfied than patients from other ethnic backgrounds. However, for assessment purposes, provided there is a reasonable mix of SPs, the effect is quite small.

The effects of task sequence on examinee performance.

BACKGROUND: If the sequence of administration of encounters in performance-based examinations has an appreciable impact on examinee performance, the validity of any associated assessment decisions could be compromised. PURPOSE: The purpose of this article was to determine if the order in which encounters occurred in a multistation standardized patient assessment had an effect on examinee performance and, if so, could this be explained by examinee experience and familiarity with the assessment. METHODS: Analysis of the scores of over 11,000 examinees who took a 10-station clinical skills assessment was performed to determine trends across the cases. RESULTS: The results showed that administration sequence does have significant, albeit small, impact on examinee performance. In general, examinees perform slightly better as they proceed through the assessment, especially in the first few encounters. CONCLUSIONS: The possible reasons for this effect were difficult to establish, but it is likely that comfort with the examination format plays a significant role. For a given assessment, all examinees saw the same cases in a fixed sequence. Therefore, as long as performance gains are relatively small and consistent, it is unlikely that sequence effects compromised the fairness of the assessment.