The Intra- and Inter-rater Reliability and the Learning Curve for a Simple Neurological Score for Rats

ABSTRACT

OBJECTIVE: To measure the intra- and inter-rater reliability of a simple sensorimotor performance test for rats, and to evaluate the learning efficiency of a novice rater for the test. METHOD: Middle cerebral arteries were occluded by intraluminal sutures in 25 male Sprague-Dawley rats (10~12 weeks old). The sensorimotor performance test was performed by a novice and an experienced rater, with each rater performing the test twice each week for 3 consecutive weeks. A ten-minute standardized video about the rating method was shown to the novice rater after the second test each week. RESULTS: The intra- and inter-rater agreement was determined using Cohen's weighted kappa coefficient. The intra-rater reliability was initially poor for the novice (κ [95% confidence interval], 0.31[-0.02, 0.64]), but it improved significantly after 3-week self education using the standardized video (0.81 [0.69, 0.93], showing almost perfect agreement. The reliability of the experienced researcher was good at all times (κ = 0.64, 0.76, 0.71, for week 1, 2, 3, respectively), indicating substantial agreement. The inter-rater reliability showed clear improvement after self-education (κ = 0.44, 0.69, 0.69, for week 1, 2, 3, respectively). Although the total sum score was highly reliable, some of the individual items showed lower intra-and inter-rater agreement. However, each rater showed greater within-rater variability for different subtests. CONCLUSION: The simple sensorimotor performance test showed high degree of intra- and inter-rater agreement when performed by experienced or properly educated raters. The inaccuracy of the novice was rectified by 3-week self-education using a video.