The Effects of Vignette Scoring on Reliability and Validity of Self-Reports.

The research presented in this article combines mathematical derivations and empirical results to investigate effects of the nonparametric anchoring vignette approach proposed by King, Murray, Salomon, and Tandon on the reliability and validity of rating data. The anchoring vignette approach aims to correct rating data for response styles to improve comparability across individuals and groups. Vignettes are used to adjust self-assessment responses on the respondent level but entail significant assumptions: They are supposed to be invariant across respondents, and the responses to vignette prompts are supposed to be without error and strictly ordered. This article shows that these assumptions are not always met and that the anchoring vignette approach leads to higher Cronbach's alpha values and increased correlations among adjusted variables regardless of whether the assumptions of the approach are met or violated. Results suggest that the underlying assumptions and effects of the anchoring vignette approach should be carefully examined as the increased correlations and reliability estimates can be observed even for response variables that are independent random draws and uncorrelated with any other variable.

The Reference Ability Neural Network Study: motivation, design, and initial feasibility analyses.

We introduce and describe the Reference Ability Neural Network Study and provide initial feasibility data. Based on analyses of large test batteries administered to individuals ranging from young to old, four latent variables, or reference abilities (RAs) that capture the majority of the variance in age-related cognitive change have been identified: episodic memory, fluid reasoning, perceptual speed, and vocabulary. We aim to determine whether spatial fMRI networks can be derived that are uniquely associated with the performance of each reference ability. We plan to image 375 healthy adults (50 per decade from age 20 to 50; 75 per decade from age 50 to 80) while performing a set of 12 cognitive tasks. Data on 174 participants are reported here. Three tasks were grouped a priori into each of the four reference ability domains. We first assessed to what extent both cognitive task scores and activation patterns readily show convergent and discriminant validity, i.e. increased similarity between tasks within the same domain and decreased similarity between tasks between domains, respectively. Block-based time-series analysis of each individual task was conducted for each participant via general linear modeling. We partialled activation common to all tasks out of the imaging data. For both test scores and activation topographies, we then calculated correlations for each of 66 possible pairings of tasks, and compared the magnitude of correlation of tasks within reference ability domains to that of tasks between domains. For the behavioral data, globally there were significantly stronger inter-task correlations within than between domains. When examining individual abilities, 3 of the domains also met these criteria but memory reached only borderline significance. Overall there was greater topographic similarity within reference abilities than between them (p<0.0001), but when examined individually, statistical significance was reached only for episodic memory and perceptual speed. We then turned to a multivariate technique, linear indicator regression analysis, to derive four unique linear combinations of Principal Components (PC) of imaging data that were associated with each RA. We investigated the ability of the identified PCs to predict the reference domain associated with the activation of individual subjects for individual tasks. Median accuracy rates for associating component task activation with a particular reference ability were quite good: memory: 82%; reasoning: 87%; speed: 84%; vocabulary: 77%. These results demonstrate that even using basic GLM analysis, the topography of activation of tasks within a domain is more similar than tasks between domains. The follow-up regression analyses suggest that all tasks with each RA rely on a common network, unique to that RA. Our ultimate goal is to better characterize these RA neural networks and then study how their expression changes across the age span. Our hope is that by focusing on these networks associated with key features of cognitive aging, as opposed to task-related activation associated with individual tasks, we will be able to advance our knowledge regarding the key brain changes that underlie cognitive aging.

O que mede o desenho da figura humana?: Estudos de validade convergente e disciminante / What the human figure drawing test measure?: Convergent and discriminant validity study

O Desenho da Figura Humana (DFH) constitui um dos instrumentos mais utilizados na prática de avaliação psicológica no Brasil. São poucos os estudos de validade como instrumento de avaliação cognitiva. O presente trabalho analisa o desempenho de crianças belo horizontinas no DFH (sistema Wechsler de correção) e nos testes Matrizes Progressivas de Raven, Escala Verbal do WISC-III, Teste R-2 e Teste de Bender (sistema Koppitz). Os resultados de correlação parcial (controlando-se a idade das amostras) apontaram baixas associações,porém significativas, entre o DFH e os testes Raven (0,204); Escala Verbal do WISC-III (0,299); R-2 (0,330) eBender (-0,396). Por outro lado, ao comparar três sistemas de correção do DFH (Wechsler, Goodenough eHarris), encontraram-se altos coeficientes de correlação entre eles (0,72 a 0,79). Conclui-se que o DFH é umamedida "proxy" (intermediária) da inteligência e que demanda, em intensidade baixa a moderada, tanto componentes cognitivos quanto psicomotores. Também, observa-se não haver diferença substancial entre os três sistemas de correção, abrindo a possibilidade de usar um ou outro sistema.

The Human Figure Drawing Test (HFD) constitutes an instrument often used in Brazilian psychological assessmentpractices. Few studies of its validity as a cognitive assessment instrument can be found. The present paperanalyses the performance of several samples of children from Belo Horizonte on the HFD (Wechsler scoringsystem) and on others tests such as Raven’s Coloured Progressive Matrices, verbal scale of WISC-III, R-2 Test,and Bender Test (Koppitz system). The results of partial correlation (controlling age) indicated low, but significantvalues, between HFD and Raven (.204); IQ verbal (.299); R-2 (.330), and Bender (-.396). In addition, threescoring systems of HFD (Wechsler, Goodenough, and Harris systems) were compared. High coefficients werefound (.72 to .79). It is concluded that the HFD is a proxy measure of intelligence and is fairly associated tocognitive and psychomotor abilities. On the other hand, there are not significant differences between the threescoring systems which means that it is possible to use either of them.

O que mede o desenho da figura humana?: Estudos de validade convergente e disciminante / What the human figure drawing test measure?: Convergent and discriminant validity study

O Desenho da Figura Humana (DFH) constitui um dos instrumentos mais utilizados na prática de avaliação psicológica no Brasil. São poucos os estudos de validade como instrumento de avaliação cognitiva. O presente trabalho analisa o desempenho de crianças belo horizontinas no DFH (sistema Wechsler de correção) e nos testes Matrizes Progressivas de Raven, Escala Verbal do WISC-III, Teste R-2 e Teste de Bender (sistema Koppitz). Os resultados de correlação parcial (controlando-se a idade das amostras) apontaram baixas associações,porém significativas, entre o DFH e os testes Raven (0,204); Escala Verbal do WISC-III (0,299); R-2 (0,330) eBender (-0,396). Por outro lado, ao comparar três sistemas de correção do DFH (Wechsler, Goodenough eHarris), encontraram-se altos coeficientes de correlação entre eles (0,72 a 0,79). Conclui-se que o DFH é umamedida "proxy" (intermediária) da inteligência e que demanda, em intensidade baixa a moderada, tanto componentes cognitivos quanto psicomotores. Também, observa-se não haver diferença substancial entre os três sistemas de correção, abrindo a possibilidade de usar um ou outro sistema.(AU)

The Human Figure Drawing Test (HFD) constitutes an instrument often used in Brazilian psychological assessmentpractices. Few studies of its validity as a cognitive assessment instrument can be found. The present paperanalyses the performance of several samples of children from Belo Horizonte on the HFD (Wechsler scoringsystem) and on others tests such as Raven’s Coloured Progressive Matrices, verbal scale of WISC-III, R-2 Test,and Bender Test (Koppitz system). The results of partial correlation (controlling age) indicated low, but significantvalues, between HFD and Raven (.204); IQ verbal (.299); R-2 (.330), and Bender (-.396). In addition, threescoring systems of HFD (Wechsler, Goodenough, and Harris systems) were compared. High coefficients werefound (.72 to .79). It is concluded that the HFD is a proxy measure of intelligence and is fairly associated tocognitive and psychomotor abilities. On the other hand, there are not significant differences between the threescoring systems which means that it is possible to use either of them.(AU)