'Hawks' and 'doves': effect of feedback on grades awarded by supervisors of student selected components.

BACKGROUND: Supervisors of some student selected components (SSCs) may appear to give higher grades than others. It is not known if feedback can influence the behaviour of supervisors in the grades they award. We have introduced feedback letters in our institution. AIMS: (1) To assess the feasibility of objectively identifying SSCs where grades awarded are consistently higher or lower than the average; (2) To assess the effect of feedback on the grades awarded by supervisors of SSCs. METHODS: The breakdown of SSC grades was examined over four consecutive years, before and after feedback letters were introduced in 2005. The grades awarded globally, and in five individual SSCs, were compared using the chi(2) goodness-of-fit test. RESULTS: (1) Individual SSCs were identified which awarded grades that were consistently different from the average. (2) Overall grades awarded in 2003/04 and 2004/05 (before feedback) were similar (chi2=0.37, df=2, p=0.83). Likewise, overall grades awarded in 2005/06 and 2006/07 (after feedback) were similar (chi2=1.72, df=2, p=0.42). Comparison of 2003/04 with 2005/06 (chi2=16.0, df=2, p<0.001), and 2006/07 (chi2=26.6, df=2, p<0.001), and of 2004/05 with 2005/06 (chi2=13.5, df=2, p=0.001), and 2006/07 (chi2=23.7, df=2, p<0.001), revealed highly significant differences. The grades awarded after feedback were higher than the grades awarded before feedback. CONCLUSIONS: The chi2 goodness-of-fit test may be used to identify individual SSCs where the grades awarded are different from the average, although the interpretation of the results thus obtained is fraught with difficulty. Our data also suggest that it is possible to influence assessors in the grades they award.

Student selected components: student-designed modules are associated with closer alignment of planned and learnt outcomes.

BACKGROUND: Student selected components (SSCs) are staff-designed modules selected by students from a menu of options provided separately from the 'core' curriculum. Students completing these do not always learn what teachers think they teach. Some medical schools also allow students to design their own modules. It is not known whether greater student input into planning of modules is associated with closer alignment of planned and learnt outcomes. AIMS: To compare student perception of learning outcomes addressed by student-designed ('self-proposed') SSCs, before and after completion, using the 'Dundee learning outcomes' template that we apply to all components of the undergraduate curriculum. METHODS: Students were required at the time of self-proposal, and subsequently as part of feedback on completed modules, to indicate which of twelve learning outcomes they felt were addressed by their self-proposed SSC. The chi2 test was used to compare student perceptions of learning outcomes before and after completion. RESULTS: More students thought that learning outcome 10 (appropriate decision making skills, clinical reasoning and judgement) was addressed after completion than before (96.3% versus 90.0%, chi2 4.99, p=0.02); for all other learning outcomes global perceptions were not significantly different after completion. Individual changes in perception ranged from 2.1% for outcome 12 (aptitude for personal development) to 19.6% for outcome 2 (competent to perform practical procedures). CONCLUSION: Greater student input into planning of modules is associated with closer alignment of planned and learnt outcomes. Our findings provide further evidence for the benefit of student-directed learning.

Student selected components: do students learn what teachers think they teach?

BACKGROUND: It is well recognized that what teachers teach and what students learn may not be the same. This applies to all parts of the undergraduate medical curriculum, but may be especially relevant to student selected components, which vary substantially in their educational content. This has not been studied previously. AIMS: To compare perceptions of students and supervisors in relation to learning outcomes addressed by student selected components, and thus to examine differences between what is taught and what is learned. METHODS: Supervisors (n = 69) were asked to indicate which of twelve learning outcomes they felt were components of teaching and assessment. Upon completion of each SSC, students were required to complete the same outcomes template as part of their feedback (n = 644). Perceptions were compared in two ways: (1) a colour-coded 'traffic-light' system was used to record agreement/disagreement between students and supervisors of individual SSCs; (2) differences in perception of outcomes across the entire SSC programme were compared using the chi(2) statistic. RESULTS: (1) The 'traffic-light' system readily identified individual SSCs where significant disagreement existed and which were subject to further scrutiny. (2) More students than supervisors thought that outcome 2 (competent to perform practical procedures) was a component of teaching and assessment (41.8% v 27.5%, chi(2) = 5.24, p = 0.02), whereas more supervisors than students thought that outcome 6 (competent in communication skills) (97.1% v 82.1%, chi(2) = 6.91, p = 0.009) and outcome 7 (competent to retrieve and handle information) (100% v 93.7%, chi(2) = 4.8, p = 0.02) were. CONCLUSIONS: Significant disagreement exists about the outcomes addressed by SSCs, suggesting that students do not always learn what teachers think they teach. The use of two complementary approaches allows global and individual comparisons to be drawn and thus provides a powerful tool to address this important issue.