Development of a written assessment for a national interprofessional cardiotocography education program.

BACKGROUND: To reduce the incidence of hypoxic brain injuries among newborns a national cardiotocography (CTG) education program was implemented in Denmark. A multiple-choice question test was integrated as part of the program. The aim of this article was to describe and discuss the test development process and to introduce a feasible method for written test development in general. METHODS: The test development was based on the unitary approach to validity. The process involved national consensus on learning objectives, standardized item writing, pilot testing, sensitivity analyses, standard setting and evaluation of psychometric properties using Item Response Theory models. Test responses and feedback from midwives, specialists and residents in obstetrics and gynecology, and medical and midwifery students were used in the process (proofreaders n = 6, pilot test participants n = 118, CTG course participants n = 1679). RESULTS: The final test included 30 items and the passing score was established at 25 correct answers. All items fitted a loglinear Rasch model and the test was able to discriminate levels of competence. Seven items revealed differential item functioning in relation to profession and geographical regions, which means the test is not suitable for measuring differences between midwives and physicians or differences across regions. In the setting of pilot testing Cronbach's alpha equaled 0.79, whereas Cronbach's alpha equaled 0.63 in the setting of the CTG education program. This indicates a need for more items and items with a higher degree of difficulty in the test, and illuminates the importance of context when discussing validity. CONCLUSIONS: Test development is a complex and time-consuming process. The unitary approach to validity was a useful and applicable tool for development of a CTG written assessment. The process and findings supported our proposed interpretation of the assessment as measuring CTG knowledge and interpretive skills. However, for the test to function as a high-stake assessment a higher reliability is required.

Unannounced in situ simulation of obstetric emergencies: staff perceptions and organisational impact.

AIM: To describe how unannounced in situ simulation (ISS) was perceived by healthcare professionals before and after its implementation, and to describe the organisational impact of ISS. STUDY DESIGN: Ten unannounced ISS involving all staff were scheduled March-August 2007. Questionnaire surveys on staff perceptions were conducted before (2003-2006) and after (2007-2008) implementation of unannounced ISS. Information from the debriefing sessions following each ISS constituted a proxy measure of the organisational impact of the ISS. RESULTS: Five out of ten of the unannounced ISS scheduled were conducted. Twenty-three members of the staff at work on a scheduled day for ISS were randomly selected to participate. Questionnaires before implementation revealed that 137/196 (70%) of staff members agreed or strongly agreed that ISS was a good idea and 52/199 (26%) thought it likely to be stressful and unpleasant. Questionnaires completed after implementation showed significantly more staff members, 135/153 (89%), thought ISS was a good idea. A significantly higher amount of staff members 50/153 (33%) found it to be stressful and unpleasant, and among midwives, 15/59 (25%) were anxious about ISS, whereas none of the obstetricians reported this. Information obtained through debriefing sessions generated learning points. CONCLUSIONS: The number of staff members with a positive perception of multiprofessional unannounced ISS increased after implementation; however, one-third considered ISS to be stressful and unpleasant and midwives more frequently so. The specific perception of ISS by each healthcare profession should be taken into account when planning ISS. The information from the debriefing sessions showed that implementation of ISS had an impact as it provided information required for organisational changes.