Children and their parents assessing the doctor-patient interaction: a rating system for doctors' communication skills.

CONTEXT: Only a patient and his or her family can judge many of the most important aspects of the doctor-patient interaction. This study evaluates the feasibility and reliability of children and their families assessing the quality of paediatricians' interactions using a rating instrument developed specifically for this purpose. METHODS: A reliability analysis using generalisability theory on the ratings from 352 doctor-patient interactions across different speciality clinics. RESULTS: Ratings were normally distributed. They were highest for 'overall' performance, and lowest for giving time to discuss the families' agenda. An appropriate sample of adults' ratings provided a reliable score (G = 0.7 with 15 raters), but children's ratings were too idiosyncratic to be reproducible (G = 0.36 with 15 raters). CONCLUSIONS AND FURTHER WORK: Accompanying adults can provide reliable ratings of doctors' interactions with children. Because an adult is usually present at the consultation their ratings provide a highly feasible and authentic approach. Sampling doctors' interactions from different clinics and with patients of both genders provides a universal picture of performance. The method is ideal to measure performance for in-training assessment or revalidation. Further work is in progress to evaluate the educational impact of feeding ratings back to the doctors being assessed, and their use in a range of clinical contexts.

Use of SPRAT for peer review of paediatricians in training.

OBJECTIVE: To determine whether a multisource feedback questionnaire, SPRAT (Sheffield peer review assessment tool), is a feasible and reliable assessment method to inform the record of in-training assessment for paediatric senior house officers and specialist registrars. DESIGN: Trainees' clinical performance was evaluated using SPRAT sent to clinical colleagues of their choosing. Responses were analysed to determine variables that affected ratings and their measurement characteristics. SETTING: Three tertiary hospitals and five secondary hospitals across a UK deanery. PARTICIPANTS: 112 paediatric senior house officers and middle grades. MAIN OUTCOME MEASURES: 95% confidence intervals for mean ratings; linear and hierarchical regression to explore potential biasing factors; time needed for the process per doctor. RESULTS: 20 middle grades and 92 senior house officers were assessed using SPRAT to inform their record of in-training assessment; 921/1120 (82%) of their proposed raters completed a SPRAT form. As a group, specialist registrars (mean 5.22, SD 0.34) scored significantly higher (t = - 4.765) than did senior house officers (mean 4.81, SD 0.35) (P < 0.001). The grade of the doctor accounted for 7.6% of the variation in the mean ratings. The hierarchical regression showed that only 3.4% of the variation in the means could be additionally attributed to three main factors (occupation of rater, length of working relationship, and environment in which the relationship took place) when the doctor's grade was controlled for (significant F change < 0.001). 93 (83%) of the doctors in this study would have needed only four raters to achieve a reliable score if the intent was to determine if they were satisfactory. The mean time taken to complete the questionnaire by a rater was six minutes. Just over an hour of administrative time is needed for each doctor. CONCLUSIONS: SPRAT seems to be a valid way of assessing large numbers of doctors to support quality assurance procedures for training programmes. The feedback from SPRAT can also be used to inform personal development planning and focus quality improvements.

Improving the quality of outpatient clinic letters using the Sheffield Assessment Instrument for Letters (SAIL).

AIM: To improve the quality of outpatient letters used as communication between hospital and primary care doctors. METHODS: On 2 separate occasions, 15 unselected outpatient letters written by each of 7 hospital practitioners were rated by another hospital doctor and a general practitioner (GP) using the Sheffield Assessment Instrument for Letters (SAIL). Individualised feedback was provided to participants following the rating of the first set of letters. The audit cycle was completed 3 months later without forewarning by repeat assessment by the same hospital and GP assessors using the SAIL tool to see if there was any improvement in correspondence. SETTING: Single centre: general paediatric outpatient department in a large district general hospital. RESULTS: All 7 doctors available for reassessment completed the audit loop, each providing 15 outpatient letters per assessment. The mean of the quality scores, derived for each letter from the summation of a 20-point checklist and a global score, improved from 23.3 (95% CI 22.1-24.4) to 26.6 (95% CI 25.8-27.4) (P = 0.001). CONCLUSIONS: The SAIL provides a feasible and reliable method of assessing the quality and content of outpatient clinic letters. This study demonstrates that it can also provide feedback with a powerful educational impact. This approach holds real potential for appraisal and revalidation, providing an effective means for the quality improvement required by clinical governance.