Prediction accuracy and financial savings of four screening tests for a sequential test of clinical performance.

BACKGROUND: Sequential testing of clinical performance is an effective strategy to reduce cost of testing. PURPOSE: To evaluate prediction accuracy and financial savings of 4 screening tests of clinical performance. METHODS: Screening tests were created from a 13-case examination taken by 434 medical students at 4 schools. Regression analysis determined prediction accuracy for 2 test outcomes. Financial savings were computed from published estimates. RESULTS: Zero false passes were obtained with the "Total Number of Cases Passed" screening test, but it saved only 27%. Sixty-two percent savings with 5% false passes occurred with the "Classification" screening test. The "Scale" and "Mini Test" screening tests would have excused 79% and 67% examinees with 5% and 1% false passes, respectively. CONCLUSIONS: Prediction accuracy varies with screening test and outcome measure. Sequential testing of clinical performance can save 40% to 60% with low false pass rates. However, programs need to consider loss of information for curriculum and individual feedback relative to financial savings.

Are medical students ready to provide HIV-prevention counseling?

PURPOSE: To determine whether medical students were prepared to assess risk and counsel patients about prevention of HIV infection, and whether HIV-related experience produced better knowledge and counseling skills. METHOD: In 1995, students at four North Carolina medical schools interviewed a standardized patient portraying a young woman concerned about HIV infection. The standardized patient recorded whether students asked risk-behavior questions and provided risk-reduction advice. A 21-item questionnaire assessed the students' knowledge of HIV testing and prevention. Students indicated whether they had had experience in educational settings related to HIV or STDs. RESULTS: 415 students completed both the patient interview and the questionnaire. Many failed to ask the patient about several HIV-risk behaviors. Although nearly all (98%) inquired about condom use, fewer than two thirds asked about the patient's history of STDs, number of sexual partners, or specific sexual practices. Most students advised the patient to use condoms. The average score on the knowledge test was 79%; 70% of students confused anonymous with confidential testing, more than half overestimated the risk of HIV transmission from a needle stick, and nearly one in ten did not know how to use a condom. Educational exposures did not produce significantly better risk assessment, counseling information, or knowledge scores. CONCLUSION: A majority of experienced medical students did not assess several important risk factors of a patient concerned about HIV infection, and many would have provided incorrect information related to HIV testing and prevention of infection. Patient contact in traditional clinical settings did not influence prevention knowledge or behavior. More innovative methods are needed to train students in HIV-infection prevention and counseling.

Development of a measure of medical faculty attitudes toward clinical evaluation of students.

PURPOSE: This research involved the development, and in particular the evaluation of the reliability, of scales to measure medical faculty attitudes toward clinical evaluation (ACE) of medical students. The intent was to create measures that yield reliable data and have practical utility in medical education research and faculty development. METHOD: A systematic, eight-step scale development protocol was used to create the instrument. In early 1993 factor analysis was used on data from 217 clinical faculty at four medical schools to refine the measures. Internal consistency and test-retest reliability analyses were performed. Analyses were also done to determine whether the attitude scores were influenced by such faculty demographic attributes as employing medical school, gender, age, tenure track status, academic rank, or academic department. RESULTS: An initial pool of 52 items was reduced to 30 items based on iterative reliability studies. Factor analysis on the 30 items yielded two scales: (1) Quality of Evaluation Procedures, 12 items, alpha = .81; and (2) Content of Departmental Evaluations, eight items, alpha = .85. Test-retest reliabilities (12 weeks) for the scales were .67 and .74, respectively. Faculty demographics did not influence attitudes about the quality of evaluation procedures. However, family physicians showed a slightly more positive attitude toward the content of departmental evaluations than did physicians in five other medical specialties. CONCLUSION: The goal of developing reliable measures of faculty attitudes toward clinical evaluation of medical students has been achieved. With baseline reliabilities established, future research should assess the validity and utility of the scales, especially in the context of clinical practice examinations.