Research on medical migration from sub-Saharan medical schools: usefulness of a feasibility process to define barriers to data collection and develop a practical study.

CONTEXT: The maldistribution of physicians in sub-Saharan Africa is having serious impacts on population health. Understanding the effect requires investigation from both donor and recipient countries. However, investigation from the perspective of donor countries has been lacking. METHODS: This brief communication describes a model process for the design of a research project that addresses medical migration issues from the perspective of eight African medical schools. During an international meeting, the participants designed an initial "ideal" study, and then rapidly tested its feasibility through a brief survey, and group discussion through a listserv, teleconferences and one face-to-face meeting. FINDINGS AND PRACTICAL IMPLICATIONS: Innovative research ideas can be followed-up with surveys to test the feasibility of an "ideal" research design, modifying the design accordingly. This is currently occurring with our medical migration survey study.

Simulations in the United States Medical Licensing Examination (USMLE).

Over the last several years there has been much attention focused on the detection and remediation of problems that pose potential threats to patient safety and that interfere with the provision of effective care. It has been noted that changes in medical education and assessment are integral to eventual improvement in this area. Within the assessment system used to licence physicians in the United States, there has been an evolution of assessment formats intended to improve the measurement of knowledge and skills, including the recent development of computer based patient simulations and clinical skills assessments. A number of new testing formats intended to further enhance assessment of critical knowledge and skills will be available in the near future.

Establishing the validity of test score inferences: performance of 4th-year U.S. medical students on the ECFMG Clinical Skills Assessment.

BACKGROUND: Although the relations among prematriculation, matriculation, and residency assessments have been studied, measures pertaining to clinical skills have rarely been incorporated in any analyses. PURPOSE: The purpose of this article was to investigate the relations between scores obtained on the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment (CSA) prototype and aptitude-ability measures designed to select medical students, to assess medical students while in medical school, and to evaluate physicians in postgraduate training programs. The sample included 122 4th-year medical students. METHODS: Relations among scores were summarized with correlation coefficients. Analysis of variance was used to compare CSA scores by departmental grades. RESULTS: Significant correlations (p < .01) were found between scores from assessments commonly used in medical school (i.e., United States Medical Licensing Exam [USMLE] Steps 1 and 2) and CSA component scores. There were weak associations between CSA measures and both Medical College Admissions Test scores and residency program evaluations. CONCLUSIONS: The relations between CSA scores and various other medically oriented ability measures provide additional evidence that inferences based on CSA scores are appropriate and valid.

Are interpersonal skills ratings influenced by gender in a clinical skills assessment using standardized patients?

PURPOSE: The purpose of this study was to explore possible performance differences in interpersonal skills (IPS) ratings as a function of candidate and standardized patient (SP) gender. METHODOLOGY: The IPS scores and SP characteristics for 79,999 patient encounters were studied. This included 18,325 (20.36%) female candidate to female SP, 26,872 (29.86%) male candidate to female SP, 18,281 (20.31%) female candidate to male SP, and 16,521 (29.47%) male candidate to male SP interactions. RESULTS: The analysis did not reveal a significant candidate gender by SP gender effect. There were no meaningful differences in IPS scores as a function of SP or candidate gender. CONCLUSIONS: The non-significant interaction between SP gender and candidate gender provides some evidence that male and female candidates are being assessed equivalently by male and female SPs. This result, combined with the extremely weak relationship between gender (candidate or SP) and IPS ratings, provides additional support for the fairness and defensibility of the IPS measures.

Validation of the doctor-patient communication component of the Educational Commission for Foreign Medical Graduates Clinical Skills Assessment.

BACKGROUND: The pivotal role of doctor-patient communication in effective health care delivery led the Educational Commission for Foreign Medical Graduates (ECFMG) to incorporate the assessment of interpersonal skills and spoken English proficiency into its Clinical Skills Assessment (CSA). Furthermore, it was decided that to pass the CSA, a candidate would need to meet or surpass defined performance standards for doctor-patient communication as a discrete component. This requirement, among others, is designed to ensure the readiness of graduates of foreign medical schools (FMGs) to enter postgraduate medical education programmes in the United States. OBJECTIVE: The primary focus of this study was to determine the extent to which performance in a simulated testing environment is related to performance in the clinical setting. METHOD: Nurses were trained to rate the communication skills of residents from the patient's perspective. A total of 43 first-year residents were evaluated. The survey ratings (n=225) were compared with the residents' CSA communication scores. RESULTS: Corrected correlations between CSA ratings and those obtained from nurses ranged from 0.61 to 0.73. CONCLUSION: This study provides evidence for the validity of the communication ratings provided by standardized patients. The reasonably strong associations between ratings obtained during testing and those obtained through observation of 'real' patient interactions suggest that external observers can provide accurate evaluations of doctor-patient communication.

Evaluating the spoken English proficiency of graduates of foreign medical schools.

PURPOSE: The purpose of this study was to gather additional evidence for the validity and reliability of spoken English proficiency ratings provided by trained standardized patients (SPs) in high-stakes clinical skills examination. METHOD: Over 2500 candidates who took the Educational Commission for Foreign Medical Graduates' (ECFMG) Clinical Skills Assessment (CSA) were studied. The CSA consists of 10 or 11 timed clinical encounters. Standardized patients evaluate spoken English proficiency and interpersonal skills in every encounter. Generalizability theory was used to estimate the consistency of spoken English ratings. Validity coefficients were calculated by correlating summary English ratings with CSA scores and other external criterion measures. Mean spoken English ratings were also compared by various candidate background variables. RESULTS: The reliability of the spoken English ratings, based on 10 independent evaluations, was high. The magnitudes of the associated variance components indicated that the evaluation of a candidate's spoken English proficiency is unlikely to be affected by the choice of cases or SPs used in a given assessment. Proficiency in spoken English was related to native language (English versus other) and scores from the Test of English as a Foreign Language (TOEFL). DISCUSSION: The pattern of the relationships, both within assessment components and with external criterion measures, suggests that valid measures of spoken English proficiency are obtained. This result, combined with the high reproducibility of the ratings over encounters and SPs, supports the use of trained SPs to measure spoken English skills in a simulated medical environment.

Predicting holistic ratings of written performance assessments from analytic scoring.

The use of experts to judge performance assessments is desirable because ratings of performances, carried out by experts in the content domain of the examination, are often considered to be the "gold standard." However, one drawback of using experts to rate performances is the high cost involved. A more economic alternative for scoring performance assessments entails using analytic scoring, which typically involves assigning points to individual traits present in the performance, and summing to arrive at a single score. This strategy is less costly, but may lack the richness of holistic scoring. This study investigates the use of regression-based techniques to predict expert judgments on a written performance task from a combination of analytic scores. Potentially, this will result in scores that approximate the richness of holistic ratings while maintaining the cost-effectiveness of analytic scoring. Results show that a substantial proportion of variance in expert judgments can be explained by the analytic scores, but that decisions based on actual expert judgments and the predicted expert judgments were not sufficiently consistent to warrant the substitution of one score for the other.

The management of patient encounter time in a high-stakes assessment using standardized patients.

OBJECTIVES: The purpose of this study was to gather information regarding the appropriateness of the length of time allotted for candidates to complete the history taking and physical examination tasks in a high-stakes standardized patient (SP) assessment. DESIGN: Data were collected on actual time used by 1548 examinees for each of their 10 standardized patient encounters, for which a maximum of 15 minutes was allotted, but not required. SETTING: The Clinical Skills Assessment Center of the Educational Commission for Foreign Medical Graduates (ECFMG), Philadelphia, Pennsylvania, USA. SUBJECTS: Graduates of foreign medical schools who are seeking ECFMG certification. RESULTS: The average time spent with the standardized patient was 13.3 minutes, suggesting that the 15-minute time limit was sufficient. A positive correlation was found between data-gathering scores and patient interview times. Candidates did tend to spend more time with SPs presenting with cases involving complex histories, as well as with cases of chronic conditions. CONCLUSIONS: Candidate time use varied as a function of type of clinical encounter, providing additional evidence of the content validity of the Clinical Skills Assessment.

Utilization of physician offices by adolescents in the United States.

UNLABELLED: Recent guidelines for adolescent primary care call for the specification of clinical services by three adolescent age subgroups. Yet analyses of office visits have either merged adolescence into one stage or divided it at age 15 years. OBJECTIVE: To explore the utilization of physician offices in the United States by early (11-14 years), middle (15-17 years), and late (18-21 years) adolescents. DESIGN: Secondary analysis of the 1994 National Ambulatory Medical Care Survey, focusing on visits made by the three adolescent age groups. SETTING: Nationally representative sample of 2426 physicians in nonfederal, nonhospital offices. SUBJECTS: A total of 33 598 visits by patients of all ages, representing 681.5 million visits in 1994. MAIN OUTCOME MEASURES: Number of visits, health insurance, providers seen, duration of visits, reasons for visits, resulting diagnoses, and counseling provided. RESULTS: Adolescents aged 11 to 21 years made 9.1% (61.8 million) of the total office visits and represented 15.4% of the total US population in 1994. This underrepresentation in visits held across all three adolescent age subgroups. Within the adolescent cohort, whites were overrepresented relative to their population proportion (78.5% of visits, 67.6% of population) and blacks and Hispanic adolescents were underrepresented (8.3% and 9.3% of visits, 15.5% and 13.1% of population). Middle adolescence signaled a life turning point from male to female predominance in office visits. Peak lifetime uninsurance rates occurred at middle adolescence for females (18.7%) and late adolescence for males (24.0%). Between childhood and early adolescence, public insurance decreased from 24.7% to 15.7% and uninsurance increased from 12.7% to 19.7%. Pediatricians accounted for the highest proportion of early adolescent visits (41.2%), family physicians for middle adolescent visits (35.3%), obstetrician-gynecologists for late adolescent female visits (37.3%), and family physicians for late adolescent male visits (34.8%). Mean visit duration during adolescence was 16 minutes, did not differ by age subgroup or sex, exceeded that of children (14.6 minutes), and was shorter than that of adults (19.3 minutes). Obstetrician-gynecologists spent more time with adolescents than did other physicians. Education or counseling was included in 50.4% of adolescent visits, ranging from 65.1% for obstetrician-gynecologists to 34.8% for internists. During early adolescence, the leading reasons for both male and female visits were respiratory (19.4%), dermatological (10.0%), and musculoskeletal (9.7%). A similar profile was found for middle and late adolescent males. For middle and late adolescent females, the leading reason for visits was special obstetrical-gynecological examination (12.8% and 21.1%), and the leading diagnosis resulting from visits was pregnancy (9.5% and 20.4%). CONCLUSIONS: Adolescents underutilize physician offices and are more likely to be uninsured than any other age group. Visits are short, and counseling is not a uniform component of care. As adolescents mature, their providers, presenting problems, and resulting diagnoses change. The data from the National Ambulatory Medical Care Survey support a staged approach to adolescent preventive services, targeted to the needs of three age subgroups.

A pediatric clinical skills assessment using children as standardized patients.

OBJECTIVES: To develop and implement a pediatric clinical skills assessment (PCSA) for residents, using children as standardized patients (SPs); to assess the psychometric adequacy of the PCSA and use it to evaluate the performance of residents; and to evaluate the feasibility of using child SPs and the response of the residents and the child SPs to participation in the PCSA. METHODS: Ten 22-minute complete patient encounters were developed, 7 with child SPs. Fifty-six residents (10 second-year pediatric residents, 29 first-year pediatric residents, and 17 first-year family practice residents) were evaluated on the following clinical skills: history taking, physical examination, interpersonal skills, and documentation and interpretation of clinical data/patient note. MAIN OUTCOME MEASURES: Patient encounter checklists, focus groups, and questionnaires. RESULTS: Average skill scores for the 56 residents were 68% (SD, 12%) for history taking, 56% (SD, 26%) for physical examination, 46% (SD, 12%) for patient note, and 68% (SD, 16%) for interpersonal skills. Second-year pediatric residents scored significantly higher on history taking than first-year pediatric and first-year family practice residents; first-year pediatric residents scored significantly higher on interpersonal skills than second-year pediatric and first-year family practice residents; and first- and second-year pediatric residents scored significantly higher on the patient note component than first-year family practice residents. All differences noted were significant at P<.05. There were no significant differences on physical examination between the groups. Reliabilities were 0.69 for history taking, 0.64 for physical examination, 0.76 for interpersonal skills, and 0.81 for the patient note component. On a Likert scale (5 indicates high; 1, low), residents rated the PCSA 3.9 for realism, 4.1 for challenge, 3.1 for enjoyment, and 2.9 for fairness. Child SPs found the experience positive. No negative effects on the children were identified by their real parents or their SP parents. CONCLUSIONS: Our development method gives content validity to our PCSA, and resident scores give indication of PCSA construct validity. Reliabilities are in the acceptable range. Residents found the PCSA challenging and realistic but less than enjoyable and fair. Use of child SPs is feasible. Resident performance scores were low relative to the performance criteria of the PCSA development group. The adequacy of clinical skills teaching and assessment in residency programs needs to be reviewed. Deficits in specific skills and overall performance of residents identified by a PCSA could be used to guide individual remediation and curricular change.

Emergency department utilization by adolescents in the United States.

BACKGROUND: Adolescents in the United States have been shown to underutilize primary care services and therefore may rely heavily on emergency service. Although several small studies have explored local emergency services for youth, there are no published reports of adolescent utilization of emergency services on a national scale. Furthermore, emergency services data have not been aggregated according to the age subgroups used by the current guidelines for adolescent care. OBJECTIVE: To explore the utilization of emergency departments in the United States by early (11 to 14 years), middle (15 to 17 years), and late (18 to 21 years) adolescent subgroups. DESIGN: Secondary analysis of the emergency department component of the 1994 National Hospital Ambulatory Medical Care Survey. SETTING: Nationally representative sample of 418 emergency departments in the United States. PATIENTS: Approximately 26,547 visits by patients of all ages, representing 93.4 million total visits in 1994 and 14.8 million adolescent visits. OUTCOME MEASURES: Number of visits, health insurance, reasons for visits, urgency of visits, resulting diagnoses, and hospitalization rates. RESULTS: Adolescents accounted for 15.4% of the population and 15.8% of emergency department visits in 1994. Late adolescents were overrepresented in emergency department visits relative to their population proportion (6.8% of visits, 5.3% of population), whereas early adolescents were underrepresented (4.6% of visits, 5.9% of population). Lack of health insurance was more common among 11- to 21-year-olds (26.2%) than either children (13.6%) or adults (22.7%). By ages 18 to 21 years, 40.5% of male visits and 27.6% of female visits were uninsured. Injury-related visits were more common among adolescents (28.6%) than either children (23.1%) or adults (18.2%). Injury was the leading reason for visits among all adolescent age-sex subgroups (36.6% to 42.0% of male visits and 14.1% to 27.2% of female visits) except females aged 18 to 21 years for whom digestive reasons ranked first (18.8%). Injury was the leading diagnosis for all adolescent age-sex subgroups, with peaks at early adolescence of 61.6% for males and 45.8% for females. Across all adolescent age-sex subgroups, 3.1% to 5.3% of visits resulted in hospitalization, and 41.0% to 52.5% of visits were urgent. These rates did not differ from those of children but were lower than those of adults. CONCLUSIONS: Utilization of emergency departments increases and health insurance decreases during adolescence, suggesting that adolescents with inadequate health insurance may rely heavily on emergency departments for their health care needs. Most adolescent visits to emergency departments are not urgent and might be better treated through nonemergency, primary care sites.

Small group estimation for public health.

We used synthetic estimation and linear regression to estimate the prevalence of selected risk factors and health status indicators in small populations. The derivation was based on the sociodemographic characteristics of the populations and the relationships between these variables and the health variables, as measured by the Ontario Health Survey (OHS). The estimates were validated by a comparison with the direct results of the OHS (gold standards). Synthetic estimates were much less dispersed than the regression estimates or the direct OHS estimates. Regression estimates performed better than synthetic estimates on most validation indicators, and combined approaches performed marginally better yet, although there were few clear patterns. Although correlation coefficients with gold standards in excess of 0.8 were obtained for some variables, the estimates rarely met pre-determined criteria for accuracy. At present these techniques have limited value for public health workers, but further work is justified, especially on approaches combining synthetic and regression estimation.

Survival of hepatitis A virus on human hands and its transfer on contact with animate and inanimate surfaces.

The survival of hepatitis A virus (HAV; strain HM175) on the hands of five volunteers was determined by depositing 10 microliters of fecally suspended virus on each fingerpad and eluting the inoculum after 0, 20, 60, 120, 180, and 240 min. The amount of virus recovered from each fingerpad at 0 min was approximately 6.0 x 10(4) PFU. At the end of 4 h, 16 to 30% of the initially recoverable virus remained detectable on the fingerpads. HAV inocula (10 microliters; approximately 1.0 x 10(4) PFU) placed on fingerpads or 1-cm-diameter metal disks were used to determine virus transfer to clean surfaces upon a 10-s contact at a pressure of nearly 0.2 kg/cm2. When the inoculum was dried for 20 min, virus transfer from fingerpad to fingerpad, fingerpad to disk, and disk to fingerpad ranged from 2,667 to 3,484 PFU, while 0 to 50 PFU could be transferred after 4 h of drying. Elevation of the contact pressure alone from 0.2 to 1.0 kg/cm2 resulted in an approximately threefold increase in the amount of virus transferred. Incorporation of friction (10 half turns of the finger during 10 s of contact) with the low and high levels of pressure gave two- and threefold increases in the PFU of virus transferred, respectively. Pressure and friction were found to significantly affect HAV transfer (F = 33.98; P less than 0.05), irrespective of the mode of transfer used. No statistically significant interaction was observed between mode of transfer and pressure or friction. The findings of this quantitative study suggest that human hands may play an important role in the direct as well as the indirect spread of HAV.

Discriminant analysis of a sexual-functioning inventory with intrafamilial and extrafamilial child molesters.

Several suggestions have been offered by clinicians who work with sex offenders concerning etiological distinctions between incest offenders and extrafamilial child molesters. In this study we hypothesized that nonincestuous child molesters would have more sexual problems in general than would incestuous offenders. Ninety-five incest offenders and 127 nonincestuous child molesters were administered the Derogatis Sexual Functioning Inventory (DSFI), a self-report instrument. On the global Sexual Functioning Index the total sample of child molesters scored below the first percentile compared to the normative sample. In a direct discriminant analysis of the 10 subscales, satisfaction, fantasy, and experience were the principle variables discriminating between incestuous and nonincestuous offenders. Incest offenders were higher on experience and satisfaction and lower on fantasy. However, the discriminant analysis failed to yield clinically relevant distinctions between the sexual problems of extrafamilial and intrafamilial child molesters. DSFI scores for both groups indicated that their overall level of sexual functioning is poor relative to the general population.