Assessing mental health presentations to the paediatric emergency Department : An inconsistent process. evaluating staff agreement using video vignettes

ABSTRACT

Background The rise in mental health presentations to Paediatric Emergency Departments (PED) during the Covid pandemic has been well documented. Whilst it has never been more important to accurately assess the mental health risk posed to young people on their arrival in PED, staff undertaking these assessments often have little formal training or oversight. Whilst physical presentations can be quantified by clinical parameters and evidenced investigations, the accurate assessment of mental health cases is inherently subjective, yet just as crucial. Objectives To assess the level of agreement between professionals routinely assessing undifferentiated mental health presentations in PED using video vignettes. Methods We identified mental health-related presentations to a tertiary PED over a one-month period. We selected six cases by random number generation, carefully altered/anonymised the details and engaged actors to re-create the presentations. Using an existing assessment matrix, participating PED and CAMHS (child and adolescent mental health service) staff watched the video vignettes and allocated presentations to 'green', 'amber' or 'red' risk categories. A free marginal multi-rater Kappa was used to assess the level of agreement between responses (0 indicating no agreement and 1 perfect agreement). Staff were asked to rate their confidence in assessing the vignettes on a modified likert scale numbered 1-5 (5 = very confident). This was undertaken as a service evaluation project after discussion with relevant Trust R&D teams. Results The Kappa for all responses was Kfree=0.21 (95%CI 0.04-0.38). Table 1 outlines Kappa values and self-reported confidence within each staff category. Conclusions When assessing the severity of mental health presentations on a simple traffic-light system, the agreement between PED staff was very low. This was lowest amongst nursing staff, who may be less familiar with the assessment matrix. Concordance was high amongst CAMHS staff, admittedly with few participants. This likely reflects their greater exposure and training. Despite the wide variation in responses, each staff group reported their collective confidence in assessing the cases similarly. This may point to a lack of standardised training and staff awareness. In order to best support young people during mental health difficulties, an accurate, objective and standardised assessment is key. This keeps young people safe, informs the level of support/ supervision they require and is crucial to de-escalate crises. This process starts in PED but practice is widely variable in our single-centre study - a level of inconsistency we would not tolerate in the assessment of physical symptoms. We plan to undertake regular multi-disciplinary training led by CAMHS to encourage standardised and robust assessments. We hope to improve the productivity and accuracy of discussions between PED and CAMHS and improve the patient journey for young people. We plan to repeat the vignettes following this intervention.