Rating of physiotherapy student clinical performance in a paediatric setting: are assessors consistent in their rating of a simulated clinical student performance?

BACKGROUND: During workplace-based clinical placements, best practice assessment states students should expect consistency between assessors rating their performance. To assist clinical educators (CEs) to provide consistent assessment of physiotherapy student performance, nine paediatric vignettes depicting various standards of simulated student performance, as assessed by the Assessment of Physiotherapy Practice (APP), were developed. The APP defines adequate on the global rating scale (GRS) as the minimally acceptable standard for an entry-level physiotherapist. The project aimed to evaluate consistency of paediatric physiotherapy educators assessing simulated student performance using the APP GRS. METHODS: Three paediatric scenarios representing neurodevelopment across three age ranges, infant, toddler and adolescent, were developed and scripted that depicted a 'not adequate', 'adequate' and 'good-excellent' performance based on the APP GRS. An expert panel (n = 9) conducted face and content validation. Once agreement was reached for all scripts, each video was filmed. A purposive sample of physiotherapists providing paediatric clinical education in Australia were invited to participate in the study. Thirty-five CEs, with minimum 3-years clinical experience and had supervised a student within the past year, were sent three videos at four-week intervals. Videos depicted the same clinical scenario, however performance varied with each video. Participants rated the performance on the four categories: 'not adequate', 'adequate', 'good' and 'excellent' Consistency among raters was assessed using percentage agreement to establish reliability. RESULTS: The vignettes were assessed a combined total of 59 times. Across scenarios, percentage agreement at the not adequate level was 100%. In contrast, the adequate scenarios for the Infant, Toddler and Adolescent video failed to meet the 75% agreement level. However, when combining adequate or good-excellent, percentage agreement was > 86%. The study demonstrated strong consensus when comparing not adequate to adequate or better performance. Importantly, no performance scripted as not adequate was passed by any assessor. CONCLUSIONS: Experienced educators demonstrate consistency in identifying not adequate from adequate or good-excellent performance when assessing a simulated student performance using the APP. Recommendation for practice: These validated video vignettes will be a valuable training tool to improve educator consistency when assessing student performance in paediatric physiotherapy.

Rating of physiotherapy student clinical performance: is it possible to gain assessor consistency?

BACKGROUND: Reliable interpretation of the Assessment of Physiotherapy Practice (APP) tool is necessary for consistent assessment of physiotherapy students in the clinical setting. However, since the APP was implemented, no study has reassessed how consistently a student performance is evaluated against the threshold standards. Therefore, the primary aim of this study was to determine the consistency among physiotherapy educators when assessing a student performance using the APP tool. METHODS: Physiotherapists (n = 153) from Australia with a minimum 3 years clinical experience and who had supervised a physiotherapy student within the past 12-months were recruited. Three levels of performance (not adequate, adequate, good/excellent) were scripted and filmed across outpatient musculoskeletal, neurorehabilitation, cardiorespiratory and inpatient musculoskeletal. In the initial phase of the study, scripts were written by academic staff and reviewed by an expert panel (n = 8) to ensure face and content validity as well as clinical relevance prior to filming. In the second phase of the study, pilot testing of the vignettes was performed by clinical academics (n = 16) from Australian universities to confirm the validity of each vignette. In the final phase, study participants reviewed one randomly allocated vignette, in their nominated clinical area and rated the student performance including a rationale for their decision. Participants were blinded to the performance level. Percentage agreement between participants was calculated for each vignette with an a priori percentage agreement of 75% considered acceptable. RESULTS: Consensus among educators across all areas was observed when assessing a performance at either the 'not adequate' (97%) or the 'good/excellent' level (89%). When assessing a student at the 'adequate' level, consensus reduced to 43%. Similarly, consensus amongst the 'not adequate' and 'good/excellent' ranged from 83 to 100% across each clinical area; while agreement was between 33 and 46% for the 'adequate' level. Percent agreement between clinical educators was 89% when differentiating 'not adequate' from 'adequate' or better. CONCLUSION: Consistency is achievable for 'not adequate' and 'good/excellent' performances, although, variability exists at an adequate level. Consistency remained when differentiating an 'adequate' or better from a 'not adequate' performance.

Graft tensioning practices in anterior cruciate ligament reconstruction amongst orthopaedic surgeons in Australia: a national survey.

PURPOSE: The application of graft tension during anterior cruciate ligament reconstruction is considered an important feature of ACLR. However, wide variation exists in relation to graft tensioning practice limiting the ability to determine the best approach. Thus, the primary aim of this study was to describe current clinical practice amongst Australian orthopaedic surgeons with respect to graft tensioning and explore influencing factors. MATERIALS AND METHODS: A survey was developed to address the aims of the study and pilot testing was completed to confirm validity and reliability. The survey population was defined as Australian orthopaedic surgeons, associated with the Australian Orthopaedic Association sub-specialty of knee to target surgeons likely to perform ACLR. The final sampling frame consisted of 192 surgeons. RESULTS: Manual tensioning was the most common method (80.5 %), with a maximum one-handed pull the most frequent description and estimated tension ranged between 41 and 60 N with the knee positioned near full extension. Surgeons using a tensioning device tended to use a higher tension (mean 81.85 N), with the knee positioned at 30° flexion (40 %). Sixteen percent reported individualising tension on viscoelasticity of the graft, graft diameter, patient anthropometry and age. Patient outcomes and available evidence were the primary factors influencing tensioning protocol. CONCLUSION: Tensioning practices appear to consist of three main approaches, (1) manual tension using a sustained maximum one-handed pull, with tension estimated as 41-60 N, applied near full extension, (2) tensioning device, mean tension of 81.85 N, at 30° knee flexion, (3) individual approach based on size and viscoelastic properties of the graft, patient anthropometry, contralateral comparison to the other knee and age of the patient.

Initial graft tension and the effect on postoperative patient functional outcomes in anterior cruciate ligament reconstruction.

PURPOSE: The aim of this review was to investigate the effect of initial graft tension on patient-specific functional outcomes after anterior cruciate ligament reconstruction and determine whether a particular tension is associated with superior functional outcome. METHODS: We performed a systematic review of prospective randomized trials with a National Health and Medical Research Council Australia level of evidence of III or higher published between 1950 and July 2012. Studies using a semitendinosus-gracilis or bone-patellar tendon-bone autograft that reported graft tension and postoperative functional outcomes were included. Quantitative analysis was performed on available data by calculating effect size (ES) both at various time points and across tensions (in Newtons). RESULTS: Initial search strategies returned 457 original publications, of which 5 articles fulfilled all exclusion and inclusion criteria. The mean score for quality was 5.8 (SD, 1.3), with 12 being the highest possible score. When compared with the preoperative side-to-side difference in anterior tibial displacement, 80 N and 78.9 N of tension recorded the largest effect at 2 weeks or less (ES, -2.98 [range, -3.82 to -2.14]) and 12 months or more (ES, -2.45 [range, -3.40 to -1.51]) postoperatively, respectively. When we compared tensions, the largest effect was toward 80 N when compared with 20 N at 2 weeks or less after surgery (ES, 0.76 [range, 0.17 to 1.35]). CONCLUSIONS: The objective of this review was to systematically assess the literature to determine whether a particular initial graft tension results in superior outcomes after anterior cruciate ligament reconstruction. From the review, there is a trend toward an initial graft tension of 78.5 to 90 N resulting in a reduced side-to-side difference in anterior laxity. However, there is insufficient evidence to conclude whether patient-specific function is improved at any specific tension. LEVEL OF EVIDENCE: Level II, systematic review of Level II studies.