Three-dimensional printing models increase inter-rater agreement for classification and treatment of proximal humerus fractures.

BACKGROUND: Proximal humerus fractures (PHF) are frequent, however, several studies show low inter-rater agreement in the diagnosis and treatment of these injuries. Differences are usually related to the experience of the evaluators and/or the diagnostic methods used. This study was designed to investigate the hypothesis that shoulder surgeons and diagnostic imaging specialists using 3D printing models and shoulder CT scans in assessing proximal humerus fractures. METHODS: We obtained 75 tomographic exams of PHF to print three-dimensional models. After, two shoulder surgeons and two specialists in musculoskeletal imaging diagnostics analyzed CT scans and 3D models according to the Neer and AO/OTA group classification and suggested a treatment recommendation for each fracture based on the two diagnostic methods. RESULTS: The classification agreement for PHF using 3D printing models among the 4 specialists was moderate (global k = 0.470 and 0.544, respectively for AO/OTA and Neer classification) and higher than the CT classification agreement (global k = 0.436 and 0.464, respectively for AO/OTA and Neer). The inter-rater agreement between the two shoulder surgeons were substantial. For the AO/OTA classification, the inter-rater agreement using 3D printing models was higher (k = 0.700) than observed for CT (k = 0.631). For Neer classification,  inter-rater agreement with 3D models was similarly higher (k = 0.784) than CT images (k = 0.620). On the other hand, the inter-rater agreement between the two specialists in diagnostic imaging was moderate. In the AO/OTA classification, the agreement using CT was higher (k = 0.532) than using 3D printing models (k = 0.443), while for Neer classification, the agreement was similar for both 3D models (k = 0.478) and CT images (k = 0.421). Finally, the inter-rater agreement in the treatment of PHF by the 2 surgeons was higher for both classifications using 3D printing models (AO/OTA-k = 0.818 for 3D models and k = 0.537 for CT images). For Neer classification, we saw k = 0.727 for 3D printing models and k = 0.651 for CT images. CONCLUSION: The insights from this diagnostic pilot study imply that for shoulder surgeons, 3D printing models improved the diagnostic agreement, especially the treatment indication for PHF compared to CT for both AO/OTA and Neer classifications On the other hand, for specialists in diagnostic imaging, the use of 3D printing models was similar to CT scans for diagnostic agreement using both classifications. TRIAL REGISTRATION: Brazil Platform under no. CAAE 12273519.7.0000.5505.

Investigating the Measurement Invariance and Method-Trait Effects of Parent and Teacher SNAP-IV Ratings of Preschool Children.

The Swanson, Nolan, and Pelham scale version IV (SNAP-IV) is widely used to assess symptoms of attention deficit hyperactivity disorder (ADHD) and oppositional defiant disorder (ODD) in children and adolescents. Nevertheless, there is insufficient data to support its use in preschool children. The study had three goals: First, to test the factorial validity of the three correlated-factors model of ADHD and ODD items of the SNAP-IV. Second, to investigate the measurement invariance of the items over time (6-month longitudinal interval) and by sex. Third, to investigate the convergent validity and method-specific influences on ADHD/ODD assessments with respect to multiple raters (parents/teachers) of children's symptoms. Participants were 618 preschool children (3.5-6 years) at baseline and 6-month follow-up. For model testing, we used confirmatory factor analysis for categorical observed variables. Method and trait effects were examined using the CT-C(M-1) model. The analyses showed partial measurement invariance over time and according to sex. Moreover, strong rater-specific effects were detected. The implication of the results for construct validation of the instrument and clinical assessment of ADHD and ODD traits are discussed.

A prospective evaluation of inter-rater agreement of routine medical records audits at a large general hospital in São Paulo, Brazil.

BACKGROUND: The quality of patient medical records is intrinsically related to patient safety, clinical decision-making, communication between health providers, and continuity of care. Additionally, its data are widely used in observational studies. However, the reliability of the information extracted from the records is a matter of concern in audit processes to ensure inter-rater agreement (IRA). Thus, the objective of this study is to evaluate the IRA among members of the Patient Health Record Review Board (PHRRB) in routine auditing of medical records, and the impact of periodic discussions of results with raters. METHODS: A prospective longitudinal study was conducted between July of 2015 and April of 2016 at Hospital Municipal Dr. Moysés Deutsch, a large public hospital in São Paulo. The PHRRB was composed of 12 physicians, 9 nurses, and 3 physiotherapists who audited medical records monthly, with the number of raters changing throughout the study. PHRRB meetings were held to reach a consensus on rating criteria that the members use in the auditing process. A review chart was created for raters to verify the registry of the patient's secondary diagnosis, chief complaint, history of presenting complaint, past medical history, medication history, physical exam, and diagnostic testing. The IRA was obtained every three months. The Gwet's AC1 coefficient and Proportion of Agreement (PA) were calculated to evaluate the IRA for each item over time. RESULTS: The study included 1884 items from 239 records with an overall full agreement among raters of 71.2%. A significant IRA increase of 16.5% (OR = 1.17; 95% CI = 1.03-1.32; p = 0.014) was found in the routine PHRRB auditing, with no significant differences between the PA and the Gwet's AC1, which showed a similar evolution over time. The PA decreased by 27.1% when at least one of the raters was absent from the review meeting (OR = 0.73; 95% CI = 0.53-1.00; p = 0.048). CONCLUSIONS: Medical record quality has been associated with the quality of care and could be optimized and improved by targeted interventions. The PA and the Gwet's AC1 are suitable agreement coefficients that are feasible to be incorporated in the routine PHRRB evaluation process.

Evaluation of a method for drug-related problems identification and classification in hospital setting: applicability and reliability.

Background Prescription evaluation by pharmacists has potential to improve pharmacotherapy management. It requires the use of robust methods to identify drug-related problems (DRP), which are important issues in pharmacotherapy. Objective To evaluate the applicability and reliability of Grupo de Investigação em Cuidados Farmacêuticos (GIGUF) method for prescription analysis, identification and classification of drug-related problems in inpatients prescriptions. Setting Department of Medical Clinic of a tertiary and teaching Brazilian hospital. Method An observational and retrospective study of identification and classification of drug-related problems. GIGUF method was used to evaluate prescriptions of hematological patients hospitalized between August and October 2015. The problems were categorized using GICUF-method classification. Three pharmacists performed inter-rater agreement analysis of the method using Kappa. Differences in prevalence of DRP was calculated by age, sex, pharmacotherapy complexity, length of stay and number of drugs. Main outcome measure (a) frequency and characteristics and (b) inter-rater agreement in identification and classification of the drug-related problems. Results A total of 211 problems were identified and 'inadequate dosing' was the most common problem. There was an association between the occurence of a drug-reklated problem and complexity of pharmacotherapy (p = 0.001) and number of drugs used (p = 0.010). The overall inter-rater agreement was moderate (k = 0.44 IC 95% 0.34-0.55) and the problem 'not suitable drug' (k = 0.55 IC 95% 0.44-0.66) had greater inter-rater agreement. Conclusion The method "Evaluation Drug Use Process" was useful for prescription analysis since it made the identification and classification of DRPs possible. The method demonstrated a moderate inter-rater agreement, and can contribute to pharmacotherapy management by hospital pharmacists.

Falamos da mesma criança? Concordância mãe - pai - professores na avaliação do temperamento de crianças portuguesas / Is that the same child? Parents' and teachers' agreement in temperament assessment of portuguese children

Com este trabalho pretendeu-se analisar o grau de concordância entre diferentes informadores - mães, pais, professores do ensino regular e dos apoios educativos (quando aplicável) - na avaliação do temperamento de crianças com 2 a 7 anos de idade. O estudo incidiu sobre uma amostra de 138 crianças, distribuídas por quatro grupos com diferentes características desenvolvimentais: síndrome de Down, Autismo, Nascimento Prematuro e Desenvolvimento Normal. Os informadores preencheram a Temperament Assessment Battery for Children - Revised nas versões para pais ou para professores. Encontraram-se correlações elevadas entre informadores que observam a criança no mesmo contexto e baixas entre avaliações de adultos pertencentes a contextos diferentes. Os resultados são discutidos em função das suas possíveis implicações para a avaliação do temperamento de crianças.(AU)

This study was intended to address the issue of convergence between informants (mother, father, regular teacher and special education teacher, whenever applicable) when assessing the temperament of children between 2 and 7 years old. The study was based on a sample of 138 children, distributed into four groups with different developmental characteristics: Down syndrome, Autism, Premature Birth, and Normal Development. Informants filled in the Temperament Assessment Battery for Children - Revised in its parents' or teachers' versions. Whereas correlations between assessments made by adults of different contexts were low, high correlations were found between informants who observe the child in the same context. Results are discussed in terms of their possible implications for the assessment of children's temperament.(AU)

Falamos da mesma criança? Concordância mãe - pai - professores na avaliação do temperamento de crianças portuguesas / Is that the same child? Parents' and teachers' agreement in temperament assessment of portuguese children

Com este trabalho pretendeu-se analisar o grau de concordância entre diferentes informadores - mães, pais, professores do ensino regular e dos apoios educativos (quando aplicável) - na avaliação do temperamento de crianças com 2 a 7 anos de idade. O estudo incidiu sobre uma amostra de 138 crianças, distribuídas por quatro grupos com diferentes características desenvolvimentais: síndrome de Down, Autismo, Nascimento Prematuro e Desenvolvimento Normal. Os informadores preencheram a Temperament Assessment Battery for Children - Revised nas versões para pais ou para professores. Encontraram-se correlações elevadas entre informadores que observam a criança no mesmo contexto e baixas entre avaliações de adultos pertencentes a contextos diferentes. Os resultados são discutidos em função das suas possíveis implicações para a avaliação do temperamento de crianças.

This study was intended to address the issue of convergence between informants (mother, father, regular teacher and special education teacher, whenever applicable) when assessing the temperament of children between 2 and 7 years old. The study was based on a sample of 138 children, distributed into four groups with different developmental characteristics: Down syndrome, Autism, Premature Birth, and Normal Development. Informants filled in the Temperament Assessment Battery for Children - Revised in its parents' or teachers' versions. Whereas correlations between assessments made by adults of different contexts were low, high correlations were found between informants who observe the child in the same context. Results are discussed in terms of their possible implications for the assessment of children's temperament.

Fidedignidade do WISC-III com base na concordância entre avaliadores / WISC-III reliability based in inter-rater agreement

Este trabalho pretendeu estabelecer os índices de fidedignidade do WISC-III, utilizando a técnica de concordância entre avaliadores, tendo em vista, que a subjetividade dos profissionais que utilizam o teste pode interferir diretamente em seus resultados. Foram selecionados, aleatoriamente, da amostra de padronização ao contexto brasileiro, protocolos dos testes de três crianças com a mesma idade, os quais foram pontuados por 42 psicólogos de diferentes estados do Brasil. Baseando-se nos escores totais, a precisão foi calculada através do índice de correlação intra-classe e os coeficientes, em geral, foram considerados fortes. Os subtestes não-verbais, com exceção de Completar Figuras, apresentaram os índices mais altos, sendo que as correlações dos subtestes verbais foram menores. Os resultados evidenciaram que a correção objetiva dos itens propicia maior consenso entre os avaliadores e quando as respostas a serem pontuadas envolvem verbalizações, existem maiores dificuldades.(AU)

This work aimed to establish Wechsler Intelligence Scale for Children - Third Edition (WISC-III) reliability indexes through the inter-rater agreement technique, keeping in mind that the scores could be influenced by subjective factors of the raters. Three WISC-III protocols of children of the same age were randomly selected from the Brazilian standardization sample and were scored by 42 psychologists from different States of Brazil. Reliability was assessed through the intraclass correlation index, on the base of total scores. Coefficients were considered strong in general. All nonverbal subtests, except Picture Completion, presented higher indexes than the verbal subtests. Results suggest that objective scoring propitiates greater consensus between the raters, and verbal answers produce more scoring difficulties.(AU)

Fidedignidade do WISC-III com base na concordância entre avaliadores / WISC-III reliability based in inter-rater agreement

Este trabalho pretendeu estabelecer os índices de fidedignidade do WISC-III, utilizando a técnica de concordância entre avaliadores, tendo em vista, que a subjetividade dos profissionais que utilizam o teste pode interferir diretamente em seus resultados. Foram selecionados, aleatoriamente, da amostra de padronização ao contexto brasileiro, protocolos dos testes de três crianças com a mesma idade, os quais foram pontuados por 42 psicólogos de diferentes estados do Brasil. Baseando-se nos escores totais, a precisão foi calculada através do índice de correlação intra-classe e os coeficientes, em geral, foram considerados fortes. Os subtestes não-verbais, com exceção de Completar Figuras, apresentaram os índices mais altos, sendo que as correlações dos subtestes verbais foram menores. Os resultados evidenciaram que a correção objetiva dos itens propicia maior consenso entre os avaliadores e quando as respostas a serem pontuadas envolvem verbalizações, existem maiores dificuldades...

This work aimed to establish Wechsler Intelligence Scale for Children - Third Edition (WISC-III) reliability indexes through the inter-rater agreement technique, keeping in mind that the scores could be influenced by subjective factors of the raters. Three WISC-III protocols of children of the same age were randomly selected from the Brazilian standardization sample and were scored by 42 psychologists from different States of Brazil. Reliability was assessed through the intraclass correlation index, on the base of total scores. Coefficients were considered strong in general. All nonverbal subtests, except Picture Completion, presented higher indexes than the verbal subtests. Results suggest that objective scoring propitiates greater consensus between the raters, and verbal answers produce more scoring difficulties...

Pais, educadores e testes: estão de acordo na avaliação de aptidões de crianças pré-escolares? / Parents, teachers and tests: do they agree in the assessment of preschoolers’ abilities?

O presente estudo pretende contribuir para o esclarecimento de questões relativas ao acordo entre pais e educadores quando avaliam o desenvolvimento infantil e com o grau em que estas avaliações estão relacionadas com aptidões cognitivas das crianças. Cento e setenta e três crianças pré-escolares foram classificadas de modo independente por pais e por educadores em diferentes áreas do desenvolvimento e os resultados na WPPSI-R foram tomados como critério externo. A idade e o género das crianças foram considerados nas análises realizadas. O grau de acordo entre informadores revelou-se baixo a moderado. Em comparação com os educadores, os pais classificam os seus filhos como mais desenvolvidos. Algumas dimensões do desenvolvimento (e.g. aprendizagem) encontram-se mais relacionadas com o desempenho na WPPSI-R do que outras. Os resultados são discutidos quanto às suas implicações para a avaliação psicológica de crianças pré-escolares.(AU)

This paper addresses issues related to the agreement between parents and teachers when they assess children's development, and to what extend those assessments are related to children's cognitive abilities. A hundred seventy three preschoolers were independently classified by their parents and teachers regarding different developmental domains. The results on the WPPSI-R test were taken as an external criterion to evaluate the accuracy of the parents' ratings. Children's age and gender were considered in the analysis. Inter-rater agreement is low to moderate. Compared with teachers, parents systematically rate their children as more developed. Some dimensions of development (e.g. learning) are more related to performance on the WPPSIR than others. Results are discussed regarding their implications to the psychological assessment of preschoolers.(AU)

Pais, educadores e testes: estão de acordo na avaliação de aptidões de crianças pré-escolares? / Parents, teachers and tests: do they agree in the assessment of preschoolers’ abilities?

O presente estudo pretende contribuir para o esclarecimento de questões relativas ao acordo entre pais e educadores quando avaliam o desenvolvimento infantil e com o grau em que estas avaliações estão relacionadas com aptidões cognitivas das crianças. Cento e setenta e três crianças pré-escolares foram classificadas de modo independente por pais e por educadores em diferentes áreas do desenvolvimento e os resultados na WPPSI-R foram tomados como critério externo. A idade e o género das crianças foram considerados nas análises realizadas. O grau de acordo entre informadores revelou-se baixo a moderado. Em comparação com os educadores, os pais classificam os seus filhos como mais desenvolvidos. Algumas dimensões do desenvolvimento (e.g. aprendizagem) encontram-se mais relacionadas com o desempenho na WPPSI-R do que outras. Os resultados são discutidos quanto às suas implicações para a avaliação psicológica de crianças pré-escolares.

This paper addresses issues related to the agreement between parents and teachers when they assess children's development, and to what extend those assessments are related to children's cognitive abilities. A hundred seventy three preschoolers were independently classified by their parents and teachers regarding different developmental domains. The results on the WPPSI-R test were taken as an external criterion to evaluate the accuracy of the parents' ratings. Children's age and gender were considered in the analysis. Inter-rater agreement is low to moderate. Compared with teachers, parents systematically rate their children as more developed. Some dimensions of development (e.g. learning) are more related to performance on the WPPSIR than others. Results are discussed regarding their implications to the psychological assessment of preschoolers.