The cultural competence of health care professionals: conceptual analysis using the results from a national pilot study of training and assessment.

The cultural competence of health professionals affects the satisfaction with, and outcomes of, patient care. Patient-centered cultural training was implemented with 76 trainees. Four months later, they were assessed using standardized patient scenarios and their performance compared to a control group. A questionnaire was used to evaluate the training. Assessment scores were analyzed using t tests to compare groups. Interrater reliability was calculated for individual scenarios. The cultural training received positive evaluations with many trainees indicating their intention to change future practice. Those who received cultural training gained higher scores upon assessment than those who had not. This difference was significant for three scenarios with nonclinical evaluators. Interrater reliability ranged from .913 to .427. The significant difference in performance between the groups supports the validity and educational impact of this approach. However, variable reliability highlights the difficulties with developing robust assessments in this area that are feasible within resource constraints.

Evidence for validity within workplace assessment: the Longitudinal Evaluation of Performance (LEP).

OBJECTIVE: The drive towards valid and reliable assessment methods for health professions' training is becoming increasingly focused towards authentic models of workplace performance assessment. This study investigates the validity of such a method, longitudinal evaluation of performance (LEP), which has been implemented in the assessment of postgraduate dental trainees in Scotland. Although it is similar in format to the mini-CEX (mini clinical evaluation exercise) and other tools that use global ratings for assessing performance in the workplace, a number of differences exist in the way in which the LEP has been implemented. These include the use of a reference point for evaluators' judgement that represents the standard expected upon completion of the training, flexibility, a greater range of cases assessed and the use of frequency scores within feedback to identify trainees' progress over time. METHODS: A range of qualitative and quantitative data were collected and analysed from 2 consecutive cohorts of trainees in Scotland (2002-03 and 2003-04). RESULTS: There is rich evidence supporting the validity, educational impact and feasibility of the LEP. In particular, a great deal of support was given by trainers for the use of a fixed reference point for judgements, despite initial concerns that this might be demotivating to trainees. Trainers were highly positive about this approach and considered it useful in identifying trainees' progress and helping to drive learning. CONCLUSIONS: The LEP has been successful in combining a strong formative approach to continuous assessment with the collection of evidence on performance within the workplace that (alongside other tools within an assessment system) can contribute towards a summative decision regarding competence.