ABSTRACT
Building on the comparative nonpatient study of Pianowski et al., we examine data from the Rorschach Performance Assessment System (R-PAS) and Comprehensive System (CS) in 100 nonpatients and 100 patients, 50 of each per system. Replicating their results but now in a patient sample, R-PAS produced more patient protocols having an optimal number of responses (R) for interpretation and eliminated the need for readministration due to low R. The R-PAS protocols were also much less variable in R, despite having about 2.5 more responses. Extending their results, we document that the primary markers of psychopathology in each system validly differentiate patients from nonpatients. However, R-PAS produced stronger effects. Finally, Complexity added to the valid discrimination of patients from nonpatients just for R-PAS, with patients producing less complex and rich records. The more erratic variability in R for the CS produced larger Complexity standard deviations (SDs) that obscured these genuine differences in people. We discuss implications for research and applied practice, along with directions for future research.
Subject(s)
Rorschach Test , Humans , Psychometrics , Reproducibility of Results , Psychopathology , PatientsABSTRACT
The study of the relationship between reasoning and emotional processes is not new in Psychology. There are currently two main approaches to understanding the aspects related to these processes called emotional intelligence: the ability model and the trait model. This study focuses on the latter, analyzing the factor structure, reliability, and validity of the Trait Emotional Intelligence Questionnaire (TEIQue) in a Brazilian sample. 4314 adults with ages ranging from 18 to 60 years (M = 30.37; SD = 9.73) answered the TEIQue and other online instruments measuring emotional regulation, impulsiveness, alexithymia, loneliness, quality of life, positive and negative affect, personality traits, emotional perception, emotional understanding, and reasoning tests. The original four-factor structure of the TEIQue was replicated, Cronbach's alphas ranged from 0.60 to 0.89 for the facets, and from 0.76 to 0.90 for the factors and global score. The correlations followed theoretically expected directions, showing a stronger pattern for self-report measures than for performance tasks. Our results corroborated previous studies with the TEIQue, confirming the psychometric adequacy of the instrument in the Brazilian context. Future studies may focus on participants with lower levels of education and additional external criteria, such as career preferences, job performance, and health.
ABSTRACT
We examine interrater reliability for scoring the Rorschach Performance Assessment System (R-PAS) in a sample of 89 adolescents (MAge = 13.2, SD = 1.01) from Brazil using exact agreement intraclass correlations coefficient (ICCs) for the 60 protocol-level scores that are the focus of interpretation. The first author completed or reviewed all of the primary coding, and seven R-PAS proficient psychologists trained at different sites independently produced secondary coding. Overall, excellent agreement was found (M ICC = 0.89; SD = 0.09). When averaged across this study and three other comparison studies, stronger reliability was present, in general, for commonly coded variables (M = 0.87) as opposed to rare or infrequent variables (M = 0.78). In addition, 78.3% of the variables showed excellent interrater reliability and an additional 20.0% had good reliability. The results also showed that the ICCs for most variables had low variability across studies, suggesting clear coding guidelines. However, variables with higher ICC variability across studies indicated domains where it would be desirable to expand guidelines with more detailed parameters. Overall, the findings indicate excellent interrater reliability for the great majority of codes and present solid grounds for future research on interrater reliability with R-PAS.
Subject(s)
Rorschach Test , Adolescent , Brazil , Humans , Observer Variation , Reproducibility of ResultsABSTRACT
We examined the impact of the changes in administration and coding introduced by the Rorschach Performance Assessment System (R-PAS) relative to the Comprehensive System (CS) on the Rorschach response process, as manifested in variables relevant to interpretation. We also examined the efficiency of each system to obtain protocols in an optimal range of responses (R) for interpretation. As hypothesized, when comparing 50 CS and 50 R-PAS nonpatient protocols, R-PAS produced many more protocols in the optimal R range (18-27) than the CS (78% vs. 24%) and it eliminated the need for re-administration, which was required for five CS protocols. As expected, R was less variable with R-PAS, as were two variables derived from it, R8910% and Complexity. In addition, as expected because of different Form Quality tables, R-PAS showed notably fewer and less variable perceptual distortions than the CS, and an increase in more conventional perceptions. The other 58 variables showed no reliable differences in means or standard deviations, though modest power precluded definitive inferences about equivalence. Overall, our results support previous findings about the benefit of R-PAS to obtain protocols in an optimal range for interpretation, while keeping the core manifestations of the response process unchanged.
Subject(s)
Personality Assessment/standards , Personality , Rorschach Test/standards , Adult , Female , Humans , Male , Psychometrics , Reproducibility of ResultsABSTRACT
Learning to code the imagery, communication, and behavior associated with Rorschach responding is challenging. Although there is some survey research on graduate students' impressions of their Rorschach training, research has not identified which coding decisions students find to be the most problematic and time-consuming. We surveyed students to identify what they struggled with most when learning coding and to quantify how difficult it is to learn how to code. Participants (n = 191) from the United States, Brazil, Denmark, Israel, and Italy rated 57 aspects of coding using a 4-point scale that encompassed both the time required to code and the subjective difficulty of doing so. Mean ratings for coding in general indicated that students considered the overall task challenging. Ratings also revealed that students struggled most with Cognitive Special Scores, Determinants, and extrapolating from the tables to code Form Quality for objects that were not specifically listed. The findings offer suggestions about how to improve the guidelines for some of the more difficult variables and where it is most necessary to focus teaching time. Taking these steps might help new students in learning the Rorschach.
Subject(s)
Education, Graduate , Psychology, Clinical/education , Rorschach Test , Students, Medical , Brazil , Female , Humans , Israel , Italy , Male , Surveys and Questionnaires , United StatesABSTRACT
Exner ( 1989 ) and Weiner ( 2003 ) identified 3 types of Rorschach codes that are most likely to contain personally relevant projective material: Distortions, Movement, and Embellishments. We examine how often these types of codes occur in normative data and whether their frequency changes for the 1st, 2nd, 3rd, 4th, or last response to a card. We also examine the impact on these variables of the Rorschach Performance Assessment System's (R-PAS) statistical modeling procedures that convert the distribution of responses (R) from Comprehensive System (CS) administered protocols to match the distribution of R found in protocols obtained using R-optimized administration guidelines. In 2 normative reference databases, the results indicated that about 40% of responses (M = 39.25) have 1 type of code, 15% have 2 types, and 1.5% have all 3 types, with frequencies not changing by response number. In addition, there were no mean differences in the original CS and R-optimized modeled records (M Cohen's d = -0.04 in both databases). When considered alongside findings showing minimal differences between the protocols of people randomly assigned to CS or R-optimized administration, the data suggest R-optimized administration should not alter the extent to which potential projective material is present in a Rorschach protocol.
Subject(s)
Personality Assessment/standards , Personality , Rorschach Test/standards , Adult , Female , Humans , Male , Mental Disorders/diagnosis , Psychometrics , Reproducibility of ResultsABSTRACT
To generate normative reference data for the Rorschach Performance Assessment System (R-PAS), modeling procedures were developed to convert the distribution of responses (R) in protocols obtained using Comprehensive System (CS; Exner 2003 ) administration guidelines to match the distribution of R in protocols obtained using R-Optimized Administration (Meyer, Viglione, Mihura, Erard, & Erdberg, 2011 ). This study replicates the R-PAS study, examining the impact of modeling R-Optimized Administration on Brazilian normative reference values by comparing a sample of 746 CS administered protocols to its counterpart sample of 343 records modeled to match R-Optimized Administration. The results were strongly consistent with the R-PAS findings, showing the modeled records had a slightly higher mean R and, secondarily, slightly higher means for Complexity and V-Comp, as well as smaller standard deviations for R, Complexity, and R8910%. We also observed 5 other small differences not observed in the R-PAS study. However, when comparing effect sizes for the differences in means and standard deviations observed in this study to the differences found in the R-PAS study, the results were virtually identical. These findings suggest that using R-Optimized Administration in Brazil might produce normative results that are similar to traditional CS norms for Brazil and similar to the international norms used in R-PAS.
Subject(s)
Personality Assessment/standards , Research Design/standards , Adult , Brazil , Female , Humans , Male , Mental Disorders/diagnosis , Multivariate Analysis , Psychometrics , Reference Values , Reproducibility of Results , Rorschach Test/standardsABSTRACT
This article describes 3 studies evaluating normative reference data for the Rorschach Comprehensive System (CS; Exner, 2003, 2007), with a particular focus on the viability of the Composite International Reference Values (CIRVs) that were compiled from 21 adult studies by Meyer, Erdberg, and Shaffer (2007). Study 1 documented how the CIRV norms are virtually identical when organized into 3 groups differentiated by the quality of their data collection effort, including an optimal group of 4 samples that relied on multiple experienced examiners and provided ongoing quality control over administration and coding. Analyses also showed that relative to the group of more optimal samples, the group of less optimal samples did not produce more variability in summary scores within or across samples or lower interrater reliability for coding. Study 2 used the existing CS reference norms to generate T scores for the CIRV means and documented how the CS norms make other samples of healthy nonpatients look psychologically impaired in multiple domains. Study 3 documented with examples from 4 different countries how 2 sets of within-country local norms produced notably different results on some variables, which compromises the ability of local norms to be used instead of the CIRVs. Taken together, the 3 studies provide support for the use of CIRVs in clinical practice as norms that are generalizable across samples, settings, languages, and cultures and that account for the natural variability that is present when clinicians and researchers contend with the ambiguity contained in the standard CS reference materials concerning the proper ways to administer and code. We conclude by urging CS users to rely on the CIRVs when making clinical inferences and to adopt alternative methods of ensuring they are following cohesively standardized administration and coding guidelines.
Subject(s)
Psychometrics , Rorschach Test/standards , Adult , Australia , Cultural Characteristics , Europe , Female , Humans , Male , Middle Aged , Multivariate Analysis , Personality , Reference Values , Reproducibility of Results , South America , United StatesABSTRACT
OBJECTIVE: To describe item and person parameters obtained with the Rasch model, one of the item response theory models, in the assessment of personality disorders based on Millon's theory. METHOD: A total of 350 people participated in the study. Age ranged from 18 to 67 years (mean ± standard deviation = 27.02±10.13), and 71.7% of the participants (n = 251) were female. Of the 350 individuals, 21.1% (n = 74) answered affirmatively about being under psychiatric treatment and taking psychiatric medications. The Personality Disorders Dimensional Inventory (PDDI), an instrument designed to assess personality disorders according to Millon's theory, was applied to all participants. Data were analyzed using the Rasch model. RESULTS: Overall, analysis with the Rasch model revealed that the PDDI has adequate psychometric properties for the assessment of personality disorders. CONCLUSION: Among the contributions of item response theory models for clinical instruments, the Rasch person-item map deserves to be highlighted as a successful attempt to improve the understanding of clinical scores obtained in response to particular test items.
ABSTRACT
OBJECTIVE: To describe item and person parameters obtained with the Rasch model, one of the item response theory models, in the assessment of personality disorders based on Millon's theory. METHOD: A total of 350 people participated in the study. Age ranged from 18 to 67 years (mean ± standard deviation = 27.02±10.13), and 71.7% of the participants (n = 251) were female. Of the 350 individuals, 21.1% (n = 74) answered affirmatively about being under psychiatric treatment and taking psychiatric medications. The Personality Disorders Dimensional Inventory (PDDI), an instrument designed to assess personality disorders according to Millon's theory, was applied to all participants. Data were analyzed using the Rasch model. RESULTS: Overall, analysis with the Rasch model revealed that the PDDI has adequate psychometric properties for the assessment of personality disorders. CONCLUSION: Among the contributions of item response theory models for clinical instruments, the Rasch person-item map deserves to be highlighted as a successful attempt to improve the understanding of clinical scores obtained in response to particular test items.
OBJETIVO: Descrever os parâmetros de itens e pessoas obtidos com o modelo de Rasch, um dos modelos da teoria de resposta ao item, na avaliação de transtornos da personalidade de acordo com a teoria de Millon. MÉTODO: Participaram do estudo 350 pessoas. A idade variou de 18 a 67 anos (média ± desvio padrão = 27.02±10.13), e 71.7% (n = 251) eram do sexo feminino. Dos 350 participantes, 21.1% (n = 74) responderam afirmativamente sobre estarem em tratamento psiquiátrico e utilizarem medicamento psiquiátrico. O Inventário Dimensional de Transtornos da Personalidade (IDTP), destinado à avaliação dos transtornos da personalidade com base na teoria de Millon, foi aplicado a todos os participantes. Os dados foram analisados por meio do modelo de Rasch. RESULTADOS: De modo geral, os resultados da análise por meio do modelo de Rasch revelaram que o IDTP tem parâmetros psicométricos adequados para a avaliação de transtornos da personalidade. CONCLUSÃO: Entre as contribuições da teoria de resposta ao item para instrumentos clínicos, o mapa de pessoas-itens do modelo Rasch merece destaque enquanto tentativa bem-sucedida de aumentar a compreensão das pontuações clínicas obtidas em resposta a determinadas questões de um teste.