Inter-rater reliability of a composite health promotion scoring system developed in Singapore

INTRODUCTION@#In view of the important role of the environment in improving population health, implementation of health promotion programmes is recommended in living and working environments. Assessing the prevalence of such community health-promoting practices is important to identify gaps and make continuous and tangible improvements to health-promoting environments. We aimed to evaluate the inter-rater reliability of a composite scorecard used to assess the prevalence of community health-promoting practices in Singapore.@*METHODS@#Inter-rater reliability for the use of the composite health promotion scorecards was evaluated in eight residential zones in the western region of Singapore. The assessment involved three raters, and each zone was evaluated by two raters. Health-promoting practices in residential zones were assessed based on 44 measurable elements under five domains - community support and resources, healthy behaviours, chronic conditions, mental health and common medical emergencies - in the composite scorecard using weighted kappa. The strength of agreement was determined based on Landis and Koch's classification method.@*RESULTS@#A high degree of agreement (almost perfect-to-perfect) was observed between both raters for the measurable elements from most domains and subdomains. An exception was observed for the community support and resources domain, where there was a lower degree of agreement between the raters for a few elements.@*CONCLUSION@#The composite scorecard demonstrated a high degree of reliability and yielded similar scores for the same residential zone, even when used by different raters.

Geração de itens e validação de conteúdo de domínios e de itens para o indivíduo com osteoartrite do joelho: um estudo de método misto / Item generation and content validation of domains with item pool for the individual with knee osteoarthritis: a mixed-method study

INTRODUÇÃO: Existem várias escalas para avaliar as percepções subjetivas e os componentes individuais em indivíduos com osteoartrite (OA) de joelho. Até o momento, não há escalas disponíveis conhecidas para medir o equilíbrio combinado entre mobilidade, AVD e QV em OA de joelho com base na Classificação Internacional de Funcionalidade, Incapacidade e Saúde (CIF). OBJETIVO: Gerar itens e domínios relacionados aos problemas enfrentados pelos indivíduos com OA de joelhos e validar o conteúdo por especialistas. MÉTODOS: Os domínios e itens foram gerados através de pesquisa bibliográfica extensa (ELS) para extrair itens relacionados a equilíbrio, mobilidade, ADL e QV em indivíduos com OA em joelhos baseados na CIF e através de entrevista aprofundada direta (EAD) em 13 pessoas com OA de joelhos e três especialistas. A validação de conteúdo dos domínios e itens gerados foi validada por 10 especialistas por meio da pesquisa Delphi online. O índice mínimo de validação de conteúdo em nível de item (I-CVI) de 0,80 foi considerado para validar os itens identificados e o índice de validação de conteúdo em nível de escala geral (S-CVI) de 0,90 foi fixado para validar os itens gerados para uso no processo de desenvolvimento da escala. RESULTADOS: Os 117 itens gerados por EAD na ELS foram inicialmente agrupados em 18 domínios. A validação de conteúdo pelo método Delphi resultou em uma diminuição para 56 itens agrupados em 14 domínios com SCVI de 0, 93. CONCLUSÃO: O conjunto abrangente de itens de deficiência, limitação de atividade e restrição de participação para indivíduos com OA de joelhos nos domínios propostos foi desenvolvido e o conteúdo validado. Esses itens são recomendados para uso no desenvolvimento de uma nova escala abrangente de índice de osteoartrite do joelho (CKOAI).

INTRODUCTION: There are several scales to evaluate subjective perceptions and individual components in individuals with knee osteoarthritis (IKOA). Till date, no scale is available to measure the combined balance, mobility, ADL and QoL in IKOA based on the International Classification of Functioning, Disability and Health (ICF). OBJECTIVE: The purpose of the study was to generate items and domains related to problems faced by IKOA and to validate the content by experts. METHODS: The domains and items were generated through extensive literature search (ELS) to extract items related to symptoms, balance, mobility, ADL and QoL in IKOA based on the International Classification of Functioning, Disability and Health (ICF) and through in-depth direct interview (IDDI) from 13 IKOA and three experts. The content validation of domains and items generated were validated by 10 experts through online Delphi survey. Minimum itemlevel content validation index (I-CVI) of 0.80 was considered to validate the identified items and the overall scale-level content validation index (S-CVI) of 0.90 was fixed to validate the generated items to use in scale development process. RESULTS: 117 items generated by IDDI and ELS were grouped under 18 domains initially. Content validation by Delphi method resulted in reduction with 56 item pool being grouped under the 14 domains with SCVI is 0.93. CONCLUSION: The comprehensive impairment, activity limitation and participation restriction item pool for IKOA under the proposed domains, have been developed and content validated. These items are recommended for their use in development of new comprehensive knee osteoarthritis index scale (CKOAI).

Interrater reliability of fitness gram test to be used as a reliable tool for assessing physical fitness for school children in Guwahati urban society of North-East India

Background: The assessment of physical fitness using a specific tool has become an important part to find out the fitness level of children and adolescence. Fitness gram is a health-related fitness test that utilizes criterion-referenced standards on health-related components. Through the years research has shown that Fitness gram has become one of the most widely used programs in the United States, though it’s used in India is not popular.Methods: Seventy school students in the age group of 5 to 14 years were included in this study and randomly allocated for physical fitness test where 62 students have completed the test. Two physiotherapists as a rater were allotted and children were tested by them with Fitnessgram test battery in a gap of 1 week. Outcome measures includes Fitness gram test battery. Statistical analysis was done by Cronbach’s Alpha value has been computed for the Interrater reliability.Results: Sixty-two students with both gender (male 56.3% and female 43.7%, age -10.12±2.72) were assessed for physical fitness using Fitness gram test battery. All the test variables showed an excellent reliability (Cronbach’s alpha =0.91-0.95).Conclusions: The Fitness gram test battery has been found to be reliable tool and can be implemented for physical fitness test for the children.

Validity And Inter-Rater Reliability Of Postural Analysis Among New Raters

@#Work posture analysis is crucial in observing and reducing work-related musculoskeletal symptoms in the workplace. However, in a developing country, new raters are commonly assigned to conduct postural analysis to save on cost. This study aims to observe the validity and inter-rater reliability (defined as the degree of agreement among different raters) among new raters of three different commonly used work posture analysis methods: Rapid Upper Limb Assessment (RULA), Rapid Entire Body Assessment (REBA), and Ovako Workload Assessment System (OWAS). Fifty industrial engineering students, divided into five groups, who received prior training about the use of the methods, participated voluntarily in this study by observing ten different working postures in five different industries: the tofu, military equipment manufacturing, automotive maintenance and service, cracker, and milk-processing industries. One ergonomics expert also observed the working postures. Validity was observed based on the correlation between new raters’ ratings and the rating of the ergonomics expert. Inter-rater reliability within one group was calculated using the percentage of agreement and kappa value. The result shows high validity of RULA, REBA, and OWAS among new raters. There are insignificant differences in the inter-rater reliability of new raters among RULA, REBA, and OWAS. The implications of the result are discussed.

Concordancia entre observadores de videos de pacientes con sospecha diagnóstica de autismo y psicosis infantiles / Inter-rater Reliability in Videos of Patients with a Suspected Diagnostic of Autism and Child Psychosis

RESUMEN Introducción: Dados la complejidad del abordaje y el tratamiento del autismo y las psicosis infantiles y el impacto que tienen en la vida del niño y su familia, es fundamental ser cautos a la hora de hacer un diagnóstico que tiene implicaciones pronosticas. Se utiliza la perspectiva psicodinámica para acceder a la posibilidad de incluir en el diagnóstico aspectos tanto patológicos como sanos y promover el potencial evolutivo de cada paciente. Objetivo: Determinar la concordancia entre evaluadores y evaluar cuantitativa y cualitativa mente las variables que influyen en el diagnóstico de pacientes con sospecha de autismo y psicosis infantiles desde la perspectiva psicodinámica. Métodos: Se realizó un estudio de concordancia entre observadores basado en la evaluación diagnóstica de videos de pacientes con sospecha de autismo o psicosis infantiles que asisten a junta diagnóstica (sesión de observación) en el Instituto de Ortopedia Infantil Roosevelt. Resultados: Se obtuvieron valores de kappa que oscilaron en general entre 0,24 y 0,50, con una fuerza de concordancia que varió de leve a moderada y K = 0,24 para la organización de personalidad. Esta manera de diagnosticar toma en consideración los aspectos tanto patológicos como saludables que constituyen la organización de personalidad. Al final se discuten las limitaciones y los puntos que tener en cuenta en los estudios posteriores. Conclusiones: Se recomienda que evalúe al niño con perturbaciones mayores un equipo interdisciplinario, y no un solo observador, pues este espacio favorece la discusión, que es crucial a la hora de establecer un diagnóstico y evita lecturas parciales del funcionamiento psíquico del paciente que consulta.

ABSTRACT Introduction: Diagnosing and treating autism and child psychoses is very difficult; these pathologies impact not only the child's life but also the family as a whole. Therefore caution is required when giving a diagnosis with prognostic implications. We use the psychodynamic perspective in order to take into consideration both pathological and healthy diagnostic aspects, and to promote evolutionary potential of each patient. Objective: To determine the inter-rater reliability and to test quantitatively and qualitati vely the variables involved in the diagnoses of patients with suspected autism and child psychoses, based upon psychodynamic concepts. Methods: An inter-rater reliability study was carried on, based upon the diagnostic evaluation of videos of patients with suspected autism or child psychoses who attended the diagnostic meeting (observation session) at the Instituto de Ortopedia Infantil Roosevelt. Results: Kappa values were obtained, ranging from .24 to .50, with a reliability force varying from slight to moderate, and K=.24 for personality organization. This type of diagnosis takes into account both the pathological and healthy aspects which make up personality orga nization. Finally, limitations and aspects that should be considered in further studies were discussed. Conclusions: The results reinforce the need to evaluate a child with major disorders in an interdisciplinary team, and but a single observer, in order to allow discussion, and debate, and therefore, avoid partial readings of the patient's psychological functioning.

Confiabilidad interevaluador de la prueba video head impulse test en estudiantes de primer año de la generación 2016 de la Escuela de Fonoaudiología de la Universidad de Valparaíso Casa Central / Inter-rater reliability of the video head impulse test in students from the School of Speech and Language therapist of the University of Valparaiso main campus class 2016

RESUMEN Introducción: El video head impulse test es un método rápido, no invasivo y cómodo para evaluar el reflejo vestíbulo ocular, el cual está siendo utilizado con mayor frecuencia en la evaluación de pacientes con alteraciones del equilibrio. Sin embargo, en Chile se evidencia una falta de estudios relacionados a la confiabilidad interevaluador de dicha prueba. Objetivo: Determinar el nivel de concordancia de los resultados de la eficiencia del reflejo vestíbulo ocular de los canales semicirculares horizontales, obtenidos mediante la aplicación de la prueba video head impulse test, por parte de 3 evaluadores. Material y método: Estudio cuantitativo, correlacional, no experimental transversal descriptivo, con muestreo no probabilístico, de participación voluntaria. La prueba video head impulse test fue aplicada por 3 examinadores a 30 estudiantes que cumplieron con los criterios de inclusión. Resultados: El método de Bland y Altman mostró gran concordancia en las mediciones realizadas entre los 3 examinadores. La correlación intraclase para la ganancia del canal semicircular horizontal derecho e izquierdo fue de 0,85 y 0,91 respectivamente y de 0,75 para el valor de asimetría. Conclusión: El estudio mostró una confiabilidad interevaluador adecuada de la prueba video head impulse test para la evaluación de los canales semicirculares horizontales.

ABSTRACT Introduction: The video head impulse test is a rapid, non-invasive and comfortable method for the evaluation of the vestibulo-ocular reflex that is being used more often in the evaluation of patients with balance disorders. In Chile there is a lack of studies related to the inter-rater reliability for this test. Aim: To determine the level of agreement on the results for the vestibulo-ocular reflex of the horizontal semicircular canals obtained by the execution of the video head impulse test by 3 raters. Material and method: Quantitative, correlational, non-experimental, cross-sectional descriptive study with a type of non-probability sampling with voluntary participation. The video head impulse test was applied by 3 raters to 30 students who fulfilled the inclusion criteria. Results: The Bland-Altman method demonstrated good agreement in the measurements made between the 3 raters. The inter-rater reliability for the gain of the left and right horizontal semicircular canals was 0.85 and 0.91 respectively and 0.75 for the asymmetry value. Conclusion: The study showed an adequate inter-rater reliability for the video head impulse test in the evaluation of the horizontal semicircular canals.

Specific algorithm method of scoring the Clock Drawing Test applied in cognitively normal elderly / Método com algoritmo específico para pontuação do teste do desenho do relógio em idosos cognitivamente normais

ABSTRACT. The Clock Drawing Test (CDT) is an inexpensive, fast and easily administered measure of cognitive function, especially in the elderly. This instrument is a popular clinical tool widely used in screening for cognitive disorders and dementia. The CDT can be applied in different ways and scoring procedures also vary. OBJECTIVE: The aims of this study were to analyze the performance of elderly on the CDT and evaluate inter-rater reliability of the CDT scored by using a specific algorithm method adapted from Sunderland et al. (1989). METHODS: We analyzed the CDT of 100 cognitively normal elderly aged 60 years or older. The CDT ("free-drawn") and Mini-Mental State Examination (MMSE) were administered to all participants. Six independent examiners scored the CDT of 30 participants to evaluate inter-rater reliability. RESULTS AND CONCLUSION: A score of 5 on the proposed algorithm ("Numbers in reverse order or concentrated"), equivalent to 5 points on the original Sunderland scale, was the most frequent (53.5%). The CDT specific algorithm method used had high inter-rater reliability (p<0.01), and mean score ranged from 5.06 to 5.96. The high frequency of an overall score of 5 points may suggest the need to create more nuanced evaluation criteria, which are sensitive to differences in levels of impairment in visuoconstructive and executive abilities during aging.

RESUMO. O Teste do Desenho do Relógio (TDR) é uma barata e rápida medida de função cognitiva, de fácil aplicação, especialmente em idosos. Este instrumento é uma ferramenta clínica muito conhecida, amplamente utilizada no rastreamento de transtornos cognitivos e demência. O TDR pode ser aplicado de diferentes formas e a sua pontuação também varia. OBJETIVO: Os objetivos deste estudo foram analisar o desempenho dos idosos no TDR e avaliar a confiabilidade inter-examinadores do TDR pontuado por um método com algoritmo específico, adaptado a partir dos critérios estabelecidos por Sunderland et al. (1989). MÉTODOS: Analisamos o TDR de 100 idosos cognitivamente saudáveis com 60 anos de idade ou mais. O TDR ("desenho livre") e o Mini-Exame do Estado Mental (MEEM) foram administrados em todos os participantes. Seis avaliadores independentes pontuaram 30 TDR para avaliar a confiabilidade inter-examinadores. RESULTADOS E CONCLUSÃO: A pontuação 5 do algoritmo proposto ("Os números em ordem inversa ou concentrados") equivalente a 5 pontos na escala original de Sunderland foi a mais frequente (53,5%). O método com algoritmo específico do TDR utilizado teve alta confiabilidade entre avaliadores (p<0,01), e a média da pontuação variou entre 5,06 e 5,96. A alta frequência de 5 pontos na pontuação geral pode sugerir a necessidade da elaboração de critérios de avaliação mais sutis, que sejam sensíveis às diferenças entre indícios de comprometimento nas habilidades visuoconstrutivas e executivas durante o envelhecimento.

Inter-rater Reliability of Wisconsin Gait Scale and Gait Abnormality Rating Scale in Hemiplegic Patients after Stroke / 中国康复理论与实践

Objective To study the inter-rater reliability of Wisconsin Gait Scale (WGS) and Gait Abnormality Rating Scale (GARS) in patients with stroke. Methods 20 hemiplegic patients were required to walk on their comfortable speed and videotaped from frontal, back-ward and lateral. The video recordings were scored with WGS and GARS by 2 experienced physical therapists. Intraclass correlation coeffi-cient (ICC) was calculated for the scores in each category and the total score. Results ICC for the WGS were 0.372~1, and were 0~0.875 for the GARS. Conclusion WGS is more appropriater to assess the gait of hemiplegic stroke patients than GARS.

Inter-rater Reliability of Wisconsin Gait Scale and Gait Abnormality Rating Scale in Hemiplegic Patients after Stroke / 中国康复理论与实践

@#Objective To study the inter-rater reliability of Wisconsin Gait Scale (WGS) and Gait Abnormality Rating Scale (GARS) in patients with stroke. Methods 20 hemiplegic patients were required to walk on their comfortable speed and videotaped from frontal, backward and lateral. The video recordings were scored with WGS and GARS by 2 experienced physical therapists. Intraclass correlation coefficient (ICC) was calculated for the scores in each category and the total score. Results ICC for the WGS were 0.372~1, and were 0~0.875 for the GARS. Conclusion WGS is more appropriater to assess the gait of hemiplegic stroke patients than GARS.

Measurement of Inter-Rater Reliability in Systematic Review / 한양의대학술지

Inter-rater reliability refers to the degree of agreement when a measurement is repeated under identical conditions by different raters. In systematic review, it can be used to evaluate agreement between authors in the process of extracting data. While there have been a variety of methods to measure inter-rater reliability, percent agreement and Cohen's kappa are commonly used in the categorical data. Percent agreement is an amount of actually observed agreement. While the calculation is simple, it has a limitation in that the effect of chance in achieving agreement between raters is not accounted for. Cohen's kappa is a more robust method than percent agreement since it is an adjusted agreement considering the effect of chance. The interpretation of kappa can be misled, because it is sensitive to the distribution of data. Therefore, it is desirable to present both values of percent agreement and kappa in the review. If the value of kappa is too low in spite of high observed agreement, alternative statistics can be pursued.

Assessing Reliability of Medical Record Reviews for the Detection of Hospital Adverse Events / 예방의학회지

OBJECTIVES: The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events. METHODS: We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detect the presence of 41 screening criteria (SC). The second stage was an implicit structured review by two physicians to identify the occurrence of adverse events from the positive cases on the SC. The inter-rater reliability of two nurses and that of two physicians were assessed. The intra-rater reliability was also evaluated by using test-retest method at approximately two weeks later. RESULTS: In 84.2% of the patient medical records, the nurses agreed as to the necessity for the second stage review (kappa, 0.68; 95% confidence interval [CI], 0.54 to 0.83). In 93.0% of the patient medical records screened by nurses, the physicians agreed about the absence or presence of adverse events (kappa, 0.71; 95% CI, 0.44 to 0.97). When assessing intra-rater reliability, the kappa indices of two nurses were 0.54 (95% CI, 0.31 to 0.77) and 0.67 (95% CI, 0.47 to 0.87), whereas those of two physicians were 0.87 (95% CI, 0.62 to 1.00) and 0.37 (95% CI, -0.16 to 0.89). CONCLUSIONS: In this study, the medical record review for detecting adverse events showed intermediate to good level of inter-rater and intra-rater reliability. Well organized training program for reviewers and clearly defining SC are required to get more reliable results in the hospital adverse event study.

Falamos da mesma criança? Concordância mãe - pai - professores na avaliação do temperamento de crianças portuguesas / Is that the same child? Parents' and teachers' agreement in temperament assessment of portuguese children

Com este trabalho pretendeu-se analisar o grau de concordância entre diferentes informadores - mães, pais, professores do ensino regular e dos apoios educativos (quando aplicável) - na avaliação do temperamento de crianças com 2 a 7 anos de idade. O estudo incidiu sobre uma amostra de 138 crianças, distribuídas por quatro grupos com diferentes características desenvolvimentais: síndrome de Down, Autismo, Nascimento Prematuro e Desenvolvimento Normal. Os informadores preencheram a Temperament Assessment Battery for Children - Revised nas versões para pais ou para professores. Encontraram-se correlações elevadas entre informadores que observam a criança no mesmo contexto e baixas entre avaliações de adultos pertencentes a contextos diferentes. Os resultados são discutidos em função das suas possíveis implicações para a avaliação do temperamento de crianças.

This study was intended to address the issue of convergence between informants (mother, father, regular teacher and special education teacher, whenever applicable) when assessing the temperament of children between 2 and 7 years old. The study was based on a sample of 138 children, distributed into four groups with different developmental characteristics: Down syndrome, Autism, Premature Birth, and Normal Development. Informants filled in the Temperament Assessment Battery for Children - Revised in its parents' or teachers' versions. Whereas correlations between assessments made by adults of different contexts were low, high correlations were found between informants who observe the child in the same context. Results are discussed in terms of their possible implications for the assessment of children's temperament.

Sensitivity, Specificity, Predictive Value and Inter-Rater Reliability of Malnutrition Screening Tools in Hospitalised Adult Patients

Nutrition screening is recommended as a first step of nutrition care to allow early identification and intervention of malnourished patients. The present study determined the validities and reliabilities of two malnutrition screening tools namely, the Malnutrition Universal Screening Tool (MUST) and Malnutrition Screening Tool (MST) among adult patients at the Hospital Kuala Lumpur. Methods: The sensitivity, specificity and predictive value of MUST and MST were conducted against the Subjective Global Assessment (SGA), anthropometric parameters including body mass index (BMI), calf circumference (CC),mid-upper arm circumference (MUAC) and energy intake. Inter-rater reliability was evaluated using kappa value (κ) to determine the level of agreement between raters. Results: A total of 151 subjects with mean age of 45.2 ± 13.7 years participated in this study. Prevalence of malnutrition according to MUST, MST and SGA was 34.4%, 33.8% and 19.9%, respectively. As compared to SGA, MUST and MST had a sensitivity of 96.6% and 93.3% respectively, whereas the specificity was 80.9% for both tools. The sensitivity and specificity of MUST against the anthropometric parameters (BMI, CC and MUAC) were between 53.8% to 88.8% and 67.4% to 69.9%, respectively. The sensitivity values for MST were between 46.1% to 63.6% and specificity values were between 64.4% to 67.6%. The inter-rater reliability of MUST was higher (substantial, mean (κ) = 0.78) than for MST (moderate, mean (κ) = 0.52). Conclusions: In conclusion, MUST was found to have similar validity levels but higher reliability result than MST. Based on our result, MUST is recommended for use in identifying adult patients who are at high risk of malnutrition. It can be used as a malnutrition screening tool but there is a need to evaluate the cost effectiveness of its implementation.

Fidedignidade do WISC-III com base na concordância entre avaliadores / WISC-III reliability based in inter-rater agreement

Este trabalho pretendeu estabelecer os índices de fidedignidade do WISC-III, utilizando a técnica de concordância entre avaliadores, tendo em vista, que a subjetividade dos profissionais que utilizam o teste pode interferir diretamente em seus resultados. Foram selecionados, aleatoriamente, da amostra de padronização ao contexto brasileiro, protocolos dos testes de três crianças com a mesma idade, os quais foram pontuados por 42 psicólogos de diferentes estados do Brasil. Baseando-se nos escores totais, a precisão foi calculada através do índice de correlação intra-classe e os coeficientes, em geral, foram considerados fortes. Os subtestes não-verbais, com exceção de Completar Figuras, apresentaram os índices mais altos, sendo que as correlações dos subtestes verbais foram menores. Os resultados evidenciaram que a correção objetiva dos itens propicia maior consenso entre os avaliadores e quando as respostas a serem pontuadas envolvem verbalizações, existem maiores dificuldades...

This work aimed to establish Wechsler Intelligence Scale for Children - Third Edition (WISC-III) reliability indexes through the inter-rater agreement technique, keeping in mind that the scores could be influenced by subjective factors of the raters. Three WISC-III protocols of children of the same age were randomly selected from the Brazilian standardization sample and were scored by 42 psychologists from different States of Brazil. Reliability was assessed through the intraclass correlation index, on the base of total scores. Coefficients were considered strong in general. All nonverbal subtests, except Picture Completion, presented higher indexes than the verbal subtests. Results suggest that objective scoring propitiates greater consensus between the raters, and verbal answers produce more scoring difficulties...

Validity of Workers’ Self-Reports. Evaluation of a Question Assessing Lifetime Exposure to Occupational Physical Activity.

Aims: In epidemiological studies exposure assessment based on questionnaires is the most cost-effective method. A question about lifetime exposure to occupational physical activity (OPA) was used in a population-based survey (part of the Copenhagen Aging and Midlife Biobank, CAMB). The aim of the study was to validate this question through a three-step process. Methodology: Firstly, the response process was studied by cognitive interviewing of 7 persons. Secondly, 64 persons participated in semi-structured interviews about their work-life, and expert judgments of exposure to OPA were compared with questionnaire-data. Exposure was 20 years of work in one of four categories of OPA: sedentary, standing and walking, moderate or high OPA. Kappa values were calculated for agreement and interpreted according to Landis and Koch’s criteria. Agreement was visualized in Bland-Altman plots. Thirdly, intra- and inter-rater reliability of expert judgments was tested. Results: Response process: The question had a complicated instruction, and the respondents found it hard to remember, categorize, and summate exposures. Semi-structured interviews: Kappa value for exposure to sedentary work was ‘substantial’ (0.71) but ‘fair’ for the other categories of OPA (0.27-0.29). Agreement between questionnaire and interview was higher in sedentary jobs and jobs with high OPA. Intra-rater reliability of expert judgments was ‘substantial’ or ‘moderate’ (0.60-0.71). Inter-rater reliability was high in sedentary jobs but lower in the more active jobs. Conclusion: Self-reports of lifetime exposure to sedentary work are valid in the CAMB cohort, whereas the validity of self-reports of exposure to high levels of occupational physical activity (OPA) are questionable. Thorough pre-testing of questions about lifetime OPA is recommended.

Pais, educadores e testes: estão de acordo na avaliação de aptidões de crianças pré-escolares? / Parents, teachers and tests: do they agree in the assessment of preschoolers’ abilities?

O presente estudo pretende contribuir para o esclarecimento de questões relativas ao acordo entre pais e educadores quando avaliam o desenvolvimento infantil e com o grau em que estas avaliações estão relacionadas com aptidões cognitivas das crianças. Cento e setenta e três crianças pré-escolares foram classificadas de modo independente por pais e por educadores em diferentes áreas do desenvolvimento e os resultados na WPPSI-R foram tomados como critério externo. A idade e o género das crianças foram considerados nas análises realizadas. O grau de acordo entre informadores revelou-se baixo a moderado. Em comparação com os educadores, os pais classificam os seus filhos como mais desenvolvidos. Algumas dimensões do desenvolvimento (e.g. aprendizagem) encontram-se mais relacionadas com o desempenho na WPPSI-R do que outras. Os resultados são discutidos quanto às suas implicações para a avaliação psicológica de crianças pré-escolares.

This paper addresses issues related to the agreement between parents and teachers when they assess children's development, and to what extend those assessments are related to children's cognitive abilities. A hundred seventy three preschoolers were independently classified by their parents and teachers regarding different developmental domains. The results on the WPPSI-R test were taken as an external criterion to evaluate the accuracy of the parents' ratings. Children's age and gender were considered in the analysis. Inter-rater agreement is low to moderate. Compared with teachers, parents systematically rate their children as more developed. Some dimensions of development (e.g. learning) are more related to performance on the WPPSIR than others. Results are discussed regarding their implications to the psychological assessment of preschoolers.

Inter-rater Reliability of Videofluoroscopic Dysphagia Scale

OBJECTIVE: To investigate the inter-rater agreement using the Videofluoroscopic Dysphagia Scale (VDS). METHOD: The present study was designed as a multicenter, single-blind trial. A Videofluoroscopic Swallowing Study (VFSS) was performed using the protocol described by J.A Logemann. Thick-fluid, pureed food, mechanically altered food, regularly textured food, and thin-fluid boluses were sequentially swallowed. Each participant received a 3 ml bolus followed by a 5 ml bolus of each food material, in the order mentioned above. All study procedures were video recorded. Discs containing these video recordings in random order were distributed to interpreters who were blinded to the participant information. The video recordings were evaluated using a standardized VDS sheet and the inter-rater reliability was calculated. RESULTS: In total, 100 patients participated in this study and 10 interpreters analyzed the findings. Inter-rater reliability was fair in terms of lip closure (kappa: 0.325), oral transit time (0.253), delayed triggering of pharyngeal swallowing (0.300), vallecular residue (0.275), laryngeal elevation (0.345), pyriform sinus residue (0.310), coating of the pharyngeal wall (0.310), and aspiration (0.393). However, other parameters of the oral phase were lower than those of the pharyngeal phase (0.06-0.153). Moreover, the summation of VDS reliability (intraclass correlation coefficient: 0.556) showed moderate agreement. CONCLUSION: VDS shows a moderate rate of agreement for evaluating the swallowing function. However, many of the parameters demonstrated a lower rate of agreement, particularly the oral phase parameters.

Inter-rater Reliability in a Clinical Performance Examination Using Multiple Standardized Patients for the Same Case / 한국의학교육

PURPOSE: The "standardization" of standardized patients (SP) is one of the most crucial factors for a successful clinical performance examination (CPX). This study aimed to examine the inter-rater reliability among SPs who portrayed the same case during a CPX. METHODS: The context was a CPX conducted under the supervision of CPX Seoul-Gyeonggi Consortium in K medical school in August 2007. K medical school ran 12 stations consisting of duplicated sets of 6 cases. In total, thirty SPs participated with 5 SPs acting each of the 6 cases. The SPs evaluated the student's performances in addition to portraying the cases. ANCOVA (analysis of covariance) was used to compare scores rated by the different SPs. The dependent variables were the case scores and the 4 subcomponent (history taking, physical examination, Clinical courtesy, and Patient-physician interaction) scores for each case; the independent variable was the SPs; and the covariate was the CPX total score. RESULTS: The Headache and Cough stations showed an acceptable level of reliability. Otherwise, Weight Loss and Facial Flushing failed to show consistent scores in all 4 subcomponents. Diarrhea and Lt. hemiparesis showed partial consistency. In terms of the subcomponents, the physical exam scores were most consistent and the patient-physician interaction scores were most inconsistent. CONCLUSION: This study tested the level of "standardization" of one set of CPX cases with mixed results. The authors hope that our results will contribute to quality assurance of CPX.

Effects of Rating Training on Inter-Rater Consistency for Developing a Dental Hygiene Clinical Rater Qualification System / 보건의료교육평가

We tried to develop itemized evaluation criteria and a clinical rater qualification system through rating training of inter-rater consistency for experienced clinical dental hygienists and dental hygiene clinical educators. A total of 15 clinical dental hygienists with 1-year careers participated as clinical examination candidates, while 5 dental hygienists with 3-year educations and clinical careers or longer participated as clinical raters. They all took the clinical examination as examinees. The results were compared, and the consistency of competence was measured. The comparison of clinical competence between candidates and clinical raters showed that the candidate group's mean clinical competence ranged from 2.96 to 3.55 on a 5-point system in a total of 3 instruments (Probe, Explorer, Curet), while the clinical rater group's mean clinical competence ranged from 4.05 to 4.29. There was a higher inter-rater consistency after education of raters in the following 4 items: Probe, Explorer, Curet, and insertion on distal surface. The mean score distribution of clinical raters ranged from 75% to 100%, which was more uniform in the competence to detect an artificial calculus than that of candidates (25% to 100%). According to the above results, there was a necessity in the operating clinical rater qualification system for comprehensive dental hygiene clinicians. Furthermore, in order to execute the clinical rater qualification system, it will be necessary to keep conducting a series of studies on educational content, time, frequency, and educator level.

Management of esophagcal varices at Suratthani Hospital.

This is a retrospective study of 279 patients with bleeding esophageal varices in the medical department of Suratthani Hospital from 1 July 1996 to 30 April 2003. All patients were treated with endoscopic interventions (endoscopic sclerotherpy and/or rubber band ligation). The patients consisted of 67% men, and had an average of 54.9 years. Underlying diseases included liver cirrhosis (98.2%) and was alcoholic cirrhosis (59.5%), Child Pugh class B and C (77.7%), and EV grade III (62.1%). Clinical bleeding was considered moderate to severe upper GI hemorrhage in 64.4%) of patients. The patients were already in a high risk group, so mortality the rate was high (22.2%). Secondary prevention of recurrent bleeding with propanolol and endoscopic intervention should be widely used in general hospitals to decrease the rate of rebleeding and increase and increase early detection of esophageal varices in high risk patients.