Improving the validity of script concordance testing by optimising and balancing items.

BACKGROUND: A script concordance test (SCT) is a modality for assessing clinical reasoning. Concerns had been raised about the plausible validity threat to SCT scores if students deliberately avoided the extreme answer options to obtain higher scores. The aims of the study were firstly to investigate whether students' avoidance of the extreme answer options could result in higher scores, and secondly to determine whether a 'balanced approach' by careful construction of SCT items (to include extreme as well as median options as model responses) would improve the validity of an SCT. METHODS: Using the paired sample t-test, the actual average student scores for 10 SCT papers from 2012-2016 were compared with simulated scores. The latter were generated by recoding all '-2' responses to '-1' and '+2' responses to '+1' for the whole and bottom 10% of the cohort (simulation 1), and scoring as if all students had chosen '0' for their responses (simulation 2). The actual average and simulated average scores in 2012 (before the 'balanced approach') were compared with those from 2013-2016, when papers had a good balance of modal responses from the expert reference panel. RESULTS: In 2012, a score increase was seen in simulation 1 in the third-year cohort, from 50.2 to 55.6% (t [10] = 4.818; p = 0.001). Since 2013, with the 'balanced approach', the actual SCT scores (57.4%) were significantly higher than scores in both simulation 1 and simulation 2 (46.7% and 23.9% respectively). CONCLUSIONS: When constructing SCT examinations, apart from the rigorous pre-examination optimisation, it is desirable to achieve a balance between items that attract extreme responses and those that attract median response options. This could mitigate the validity threat to SCT scores, especially for the low-performing students who have previously been shown to only select median responses and avoid the extreme responses.

Educating for collaborative practice: an interpretation of current achievements and thoughts for future directions.

CONTEXT: The Edinburgh Declaration, developed in 1998 as a pledge to alter the character of medical education to more effectively meet the needs of society, included a recommendation to increase the opportunity for joint learning between health and health-related professions, as part of the training for teamwork. This article acknowledges achievements since the Declaration in relation to this recommendation, using an umbrella term for the phenomenon, 'educating for collaborative practice', and presents a perspective framed as a series of questions to encourage reflection on future directions. METHODS: A literature interpretation, informed by philosophical hermeneutics, was conducted using text sets comprising reports and reviews from a section of the international literature since 1988. The interpretation involved: engaging with meanings as presented in the chosen texts; making iterative returns to the texts to explore emerging understanding; and ensuring parts of our understanding from particular texts were fused with complete understanding of the texts as a whole. A lens of appreciative inquiry facilitated acknowledgement of what has been achieved, while being curious about how it could be. RESULTS: Interpretation of the selected literature revealed notable achievements. Areas for further consideration were identified in relation to three themes: establishing shared understanding AND purpose behind use of terminology; being a conduit AND sharing responsibility for change; exploring ways of doing things AND ensuring ongoing inclusivity. CONCLUSIONS: Interpreting the current literature on 'educating for collaborative practice' has generated questions for reflection on how it may be otherwise. Readers are encouraged to embrace the tensions inherent in unanswered questions, providing space for communication, initiative and diversity of thought. An ongoing dialogue with the literature is proposed, asking whether educating students for a collective identity in settings where they are learning for and with patients is likely to advance educating for patient-centred collaborative practice.

Longitudinal integrated clerkships.

There is increased interest in longitudinal integrated clerkships (LICs) due to mounting evidence of positive outcomes for students, patients and supervising clinicians. Emphasizing continuity as the organizing principle of an LIC, this article reviews evidence and presents perspectives of LIC participants concerning continuity of care, supervision and curriculum, and continuity with peers and systems of care. It also offers advice on implementing or evaluating existing LIC programs.

Improving assessment practice through cross-institutional collaboration: An exercise on the use of OSCEs.

BACKGROUND: This study was undertaken to improve assessment practice on OSCEs through collaboration across geographically dispersed medical schools in Australia. METHODS: A total of eleven OSCE stations were co-developed by four medical schools and used in summative 2011 and 2012 examinations for the assessment of clinical performance in the early clinical and exit OSCEs in each school's medical course. Partial Credit Rasch Model was used to evaluate the psychometric properties of the shared OSCE data. Evaluation of the quality assurance reports was used to determine the beneficial impact of the collaborative benchmarking exercise on learning and teaching outcomes. RESULTS: The data for each examination demonstrated sufficient fit to the Rasch model with infit mean square values ranging from 0.88 to 0.99. Person separation (1.25-1.63) indices indicated good reliability. Evaluation of perceived benefits showed that the benchmarking process was successful as it highlighted common curriculum areas requiring specific focus and provided comparable data on the quality of teaching at the participating medical schools. CONCLUSION: This research demonstrates the validity of the psychometric data and benefits of evaluating clinical competence across medical schools without the enforcement of a prescriptive national curriculum or assessment.

The telehealth skills, training, and implementation project: an evaluation protocol.

BACKGROUND: Telehealth appears to be an ideal mechanism for assisting rural patients and doctors and medical students/registrars in accessing specialist services. Telehealth is the use of enhanced broadband technology to provide telemedicine and education over distance. It provides accessible support to rural primary care providers and medical educators. A telehealth consultation is where a patient at a general practice, with the assistance of the general practitioner or practice nurse, undertakes a consultation by videoconference with a specialist located elsewhere. Multiple benefits of telehealth consulting have been reported, particularly those relevant to rural patients and health care providers. However there is a paucity of research on the benefits of telehealth to medical education and learning. OBJECTIVE: This protocol explains in depth the process that will be undertaken by a collaborative group of universities and training providers in this unique project. METHODS: Training sessions in telehealth consulting will be provided for participating practices and students. The trial will then use telehealth consulting as a real-patient learning experience for students, general practitioner trainees, general practitioner preceptors, and trainees. RESULTS: Results will be available when the trial has been completed in 2015. CONCLUSIONS: The protocol has been written to reflect the overarching premise that, by building virtual communities of practice with users of telehealth in medical education, a more sustainable and rigorous model can be developed. The Telehealth Skills Training and Implementation Project will implement and evaluate a theoretically driven model of Internet-facilitated medical education for vertically integrated, community-based learning environments.