ABSTRACT
OBJECTIVE: To describe the results of two reliability studies and to assess the effect of training on interrater reliability scores. DESIGN: The first study (1) examined interrater and intrarater reliability scores (weighted and unweighted kappas) and (2) compared interrater reliability scores before and after training on the use of the Cleft Audit Protocol for Speech-Augmented (CAPS-A) with British English-speaking children. The second study examined interrater and intrarater reliability on a modified version of the CAPS-A (CAPS-A Americleft Modification) with American and Canadian English-speaking children. Finally, comparisons were made between the interrater and intrarater reliability scores obtained for Study 1 and Study 2. PARTICIPANTS: The participants were speech-language pathologists from the Americleft Speech Project. RESULTS: In Study 1, interrater reliability scores improved for 6 of the 13 parameters following training on the CAPS-A protocol. Comparison of the reliability results for the two studies indicated lower scores for Study 2 compared with Study 1. However, this appeared to be an artifact of the kappa statistic that occurred due to insufficient variability in the reliability samples for Study 2. When percent agreement scores were also calculated, the ratings appeared similar across Study 1 and Study 2. CONCLUSION: The findings of this study suggested that improvements in interrater reliability could be obtained following a program of systematic training. However, improvements were not uniform across all parameters. Acceptable levels of reliability were achieved for those parameters most important for evaluation of velopharyngeal function.
Subject(s)
Cleft Palate/complications , Speech Disorders/diagnosis , Speech Disorders/therapy , Speech-Language Pathology/education , Disability Evaluation , Female , Humans , Male , Phonetics , Reproducibility of Results , Speech Production MeasurementABSTRACT
OBJECTIVE: To achieve consistency and uniformity in reporting speech outcomes in individuals born with cleft palate with or without cleft lip using perceptual parameters that characterize their speech production behavior regardless of the language or languages spoken. DESIGN: A working group of six individuals experienced in speech and cleft palate was formed to develop a system of universal parameters for reporting speech outcomes in individuals born with cleft palate. The system was adopted in conjunction with a workshop held in Washington, D.C., that was devoted to developing the universal system. The system, which was refined further following the workshop, involves a three-stage plan consisting of (1) evaluation, (2) mapping, and (3) reporting. The current report focuses primarily on the third stage, reporting speech outcomes. RESULTS: A set of five universal speech parameters has been devised for the reporting stage. These consist of (1) hypernasality, (2) hyponasality, (3) audible nasal air emission and/or nasal turbulence, (4) consonant production errors, and (5) voice disorder. Also included are speech understandability and speech acceptability, global parameters that can be reported for any type of speech disorder. The parameters are described in detail, and guidelines for speech-sampling content and scoring procedures in relation to the parameters are presented. CONCLUSION: A plan has been developed to document speech outcomes in individuals with cleft palate, regardless of the spoken language, using a set of five universal reporting parameters and two global speech parameters.
