Item response theory and the measurement of psychiatric constructs: some empirical and conceptual issues and challenges.

Item response theory (IRT) measurement models are now commonly used in educational, psychological, and health-outcomes measurement, but their impact in the evaluation of measures of psychiatric constructs remains limited. Herein we present two, somewhat contradictory, theses. The first is that, when skillfully applied, IRT has much to offer psychiatric measurement in terms of scale development, psychometric analysis, and scoring. The second argument, however, is that psychiatric measurement presents some unique challenges to the application of IRT - challenges that may not be easily addressed by application of conventional IRT models and methods. These challenges include, but are not limited to, the modeling of conceptually narrow constructs and their associated limited item pools, and unipolar constructs where the expected latent trait distribution is highly skewed.

Evaluating commercial image registration packages for radiotherapy treatment planning.

Multi-modality imaging is involved in almost all oncology applications from diagnosis through treatment planning and follow-up. Commercial image fusion software packages are becoming available but require comprehensive evaluation to ensure reliability of fusion and the underpinning registration algorithm. This is especially critical for target volume delineation in radiotherapy. The present work seeks to assess such accuracy for a number of available registration methods. A National Electrical Manufacturers Association (NEMA) body phantom was used in evaluating computer tomography (CT), magnetic resonance (MR) and PET images. In addition, discussion is provided concerning the choice and geometry of fiducial markers in phantom studies and the effect of window level on target size, in particular in regard to the application of multimodality imaging in treatment planning. In general, the accuracy of fusion of multi-modality images was within 0.5-1.5mm of actual feature diameters and <2 ml volume of actual values, particularly in CT images.

Dose-position and dose-volume histogram analysis of standard wedged and intensity modulated treatments in breast radiotherapy.

The aim of this work was to evaluate the positional distribution of dose in a concise manner and to analyse dose-histogram results in tangential breast radiotherapy in 300 patients, randomized to standard wedged or intensity modulated radiotherapy (IMRT), for future correlation with clinical outcome data. A simple method for analysing the dose-position relationship in the treatment volume was used to compare the spatial distribution of dose in patients. The breast was divided into equal thirds (upper, middle and lower) and dose was assessed using three dose bands; 95-105%, >105-110% and >110% of the prescription dose. The effect of using IMRT on the dosimetry was assessed from dose-volume histogram data using the following parameters: percentage of the target volume receiving a dose less than 95%, greater than 105%, either less than 95% or greater than 105% of that prescribed; the mean dose; and the maximum dose. Doses greater than 105% were predominantly in the upper and lower regions of the breast in the standard wedged treatment. 96% of these patients received doses greater than 105% in the upper region of the breast and 70% received doses greater than 105% in the lower breast. Only 4% of patients allocated IMRT received doses greater than 105% in either region. Analysis of dose-volume histogram data showed that IMRT reduced the volume receiving a dose greater than 105% by a mean of 10.7% (p= or <0.001); the mean change in the volume receiving a dose less than 95% was 0.2% (p=0.63). Average mean plan dose was 101.6% for standard treatment and 99.6% for IMRT (p<0.001 for each compared with 100.0% ideal). The mean value of maximum dose was reduced from 111% to 106% in the group of patients randomized to IMRT. A simple method for describing the relationship between dose and position in the breast, which is helpful for the effective correlation of dosimetry and clinical effects, is reported. Further, application of IMRT to the tangential field irradiation of the breast has been demonstrated to reduce high dose regions in both volume and dose level without compromising either minimum dose coverage or mean dose delivered to the breast.

Factor analysis and scale revision.

This article reviews methodological issues that arise in the application of exploratory factor analysis (EFA) to scale revision and refinement. The authors begin by discussing how the appropriate use of EFA in scale revision is influenced by both the hierarchical nature of psychological constructs and the motivations underlying the revision. Then they specifically address (a) important issues that arise prior to data collection (e.g., selecting an appropriate sample), (b) technical aspects of factor analysis (e.g., determining the number of factors to retain), and (c) procedures used to evaluate the outcome of the scale revision (e.g., determining whether the new measure functions equivalently for different populations).

Item response theory and health outcomes measurement in the 21st century.

Item response theory (IRT) has a number of potential advantages over classical test theory in assessing self-reported health outcomes. IRT models yield invariant item and latent trait estimates (within a linear transformation), standard errors conditional on trait level, and trait estimates anchored to item content. IRT also facilitates evaluation of differential item functioning, inclusion of items with different response formats in the same scale, and assessment of person fit and is ideally suited for implementing computer adaptive testing. Finally, IRT methods can be helpful in developing better health outcome measures and in assessing change over time. These issues are reviewed, along with a discussion of some of the methodological and practical challenges in applying IRT methods.

Evaluating the equivalence of health care ratings by whites and Hispanics.

PURPOSE: This study was designed to assess the equivalence of a health care ratings scale administered to non-Hispanic white and Hispanic survey respondents. METHODS: We sent 18,840 questionnaires to a random sample of patients receiving medical care from a physician group association concentrated in the western United States; 7,093 were returned (59% adjusted response rate). Approximately 90% of survey respondents self-identified as white/Caucasian (n = 5,508) or Hispanic/Latino (n = 713). Interpersonal and technical aspects of medical care were assessed with 9 items, all administered with a 7-point response format: the best, excellent, very good, good, fair, poor, and very poor, with a "not applicable" option. Item response theory procedures were used to test for differential item functioning between white and Hispanic respondents. RESULTS: Hispanics were found to be significantly more dissatisfied with care than whites (effect size=0.27; P <0.05). Of the 9 test items, 2 had statistically significant differential item functioning (P <0.05): reassurance and support offered by your doctors and staff and quality of examinations received. However, summative scale scores and test characteristic curves for whites and Hispanics were similar whether or not these items were included in the scale. CONCLUSIONS: Despite some differences in item functioning, valid satisfaction-with-care comparisons between whites and Hispanics are possible. Thus, disparities in satisfaction ratings between whites and Hispanics should not be ascribed to measurement bias but should be viewed as arising from actual differences in experiences with care.

Computerization and adaptive administration of the NEO PI-R.

This study asks, how well does an item response theory (IRT) based computerized adaptive NEO PI-R work? To explore this question, real-data simulations (N = 1,059) were used to evaluate a maximum information item selection computerized adaptive test (CAT) algorithm. Findings indicated satisfactory recovery of full-scale facet scores with the administration of around four items per facet scale. Thus, the NEO PI-R could be reduced in half with little loss in precision by CAT administration. However, results also indicated that the CAT algorithm was not necessary. We found that for many scales, administering the "best" four items per facet scale would have produced similar results. In the conclusion, we discuss the future of computerized personality assessment and describe the role IRT methods might play in such assessments.

Using Multilevel Logistic Regression to Evaluate Person-Fit in IRT Models.

I describe how multilevel logistic regression can be used to assess the consistency of an individual's response pattern with an item response theory measurement model. Specifically, by treating item responses as being nested within individuals, multilevel logistic regression is used to estimate a person-response curve that models how an individual's item endorsement rate decreases as a function of item difficulty. The slope of an individual's person-response curve is used as an indicator of the degree of response consistency or person-fit. I argue that the proposed multilevel modeling approach to person-fit assessment has several potential advantages over traditional techniques. The most important advantage being that the multilevel modeling approach allows explanatory variables to be entered into the model so that the causes of response inconsistency or differential test functioning can be investigated.

Clinical implementation of a computer controlled milling machine for compensating filter production.

The procedures required for the clinical implementation of a computer controlled milling machine for producing compensators for breast radiotherapy are described. Moulds are cut in a rigid polymer foam block and filled with stainless-steel granulate. Quality assurance procedures are described for ensuring that the compensators produced are consistent and accurate. Relative and absolute dosimetric measurements are presented showing that the compensators are accurate to better than 1% and demonstrate the technique to be clinically acceptable.

Gender differences on negative affectivity: an IRT study of differential item functioning on the Multidimensional Personality Questionnaire Stress Reaction Scale.

Item response theory methods were used to study differential item functioning (DIF) between gender groups on a measure of stress reaction. Results revealed that women were more likely to endorse items describing emotional vulnerability and sensitivity, whereas men were more likely to endorse items describing tension, irritability, and being easily upset. Item factor analysis yielded 5 correlated factors, and the DIF analysis, in turn, revealed differential gender mean differences on these factors. This finding illustrates how even in an essentially unidimensional scale, comparison of group mean differences can be affected by multidimensionality caused by item clusters that share similar content. Results do not support arguments that measures of negative affective dispositions "artificially" produce gender mean differences by focusing on specific selected content areas.

Practical implementation of compensators in breast radiotherapy.

BACKGROUND AND PURPOSE: A method of using electronic portal imaging to design compensators for tangential breast irradiation has been developed. We describe how this has been implemented. MATERIALS AND METHODS: The compensator design method generates wedged and unwedged beam weights, in conjunction with templates for multiple lead-sheet compensators and pseudo-CT outlines. The latter describe the breast and lung profiles in a set of transverse slices. The layers of the compensator and pseudo-CT outlines are transferred to a treatment planning system for verification. The accuracy of the planning system for the high transmission blocks used to describe the compensators has been verified using a plotting tank system. Dose volume histogram data and transaxial and sagittal plan slices have been compared for both standard and compensated treatments for a sample set of five patients. RESULTS: The planning system predicted the dose at depths of 1.5 and 5 cm to within 2% for the compensators tested. The biggest source of discrepancy was a consequence of the planning system requiring blocks to have integer percentage transmission. For all patients studied, the compensated treatment resulted in a significant reduction in the percentage volume outside the 95-105% dose, with an average reduction of 10.2%. The percentage volume outside the 95-107% dose was also reduced by typically 3.4%. The implementation was found to yield a convenient automatic method of designing compensators using electronic portal imaging and verifying the results using a planning system. CONCLUSIONS: These results indicate that this method of implementation can be used in practice. The dosimetric accuracy of the treatment planning system is limited by the requirement that blocks should be of integer transmission, but this effect is small.

A California Q-set alexithymia prototype and its relationship to ego-control and ego-resiliency.

The primary purposes of the present study were to use the Q-sort method to develop a measure of alexithymia and to locate the construct within a two-dimensional (ego-control and ego-resiliency) model of personality. Thirteen professional judges described the characteristics of the alexithymic personality with the 100-item California Q-set. Scores from the sorts were aggregated to form the Alexithymia Prototype, which had a Spearman-Brown reliability of 0.99. Alexithymic people were described as having difficulties experiencing and expressing emotion, lacking imagination, and being literal, socially conforming, and utilitarian; they lack insight, are humorless, and experience meaninglessness; and anxiety and tension find outlet in bodily symptoms. This description is consistent, for the most part, with modern formulations of the alexithymia construct. In the language of the two-dimensional personality model, alexithymic individuals appear to be overcontrolling and lacking ego-resiliency (i.e., constricted, anxious, rigid, and withdrawn). We, therefore, compared the Alexithymia Prototype with two independently developed prototypes, Overcontrol and Ego-Resiliency. The Q-correlations between alexithymia and overcontrol and between alexithymia and ego-resiliency were 0.45 and -0.70, respectively. Although item analyses confirmed moderate overlap between alexithymia and overcontrol and considerable overlap between alexithymia and lacking ego-resiliency (ego-brittle), item differences suggest that alexithymia, indeed, is a unique personality construct.

Structure of the twenty-item Toronto Alexithymia Scale.

The 20-item Toronto Alexithymia Scale (TAS-20) purportedly measures 3 intercorrelated dimensions of the alexithymia construct: (a) difficulties identifying feelings (DIF), (b) difficulties describing feelings (DDF), and (c) externally oriented thinking (EOT). The primary purpose of this study was to evaluate the factor structure of the TAS-20 with full-information item factor analysis in 2 diverse samples: medical students (n = 219) and psychiatric (psychoactive substance dependent-abusing) inpatients (n = 204). In each group, we estimated the parameters of 1-, 2-, 3-, and 4-factor models. None of the various factor solutions resulted in clean, simple structures in either sample. In the student sample, the data were best represented (although not well) by a 3-dimensional model: DIF, DDF, and EOT. In the psychiatric sample, however, the 3-dimensional solution was quite different; DIF and DDF formed 1 "emotional awareness deficits" factor, and EOT split into 2 unrelated factors. We concluded that alexithymia, as measured by the TAS-20, is multidimensional and not well-represented by a global severity score. Our recommendations include checking the factor structure of the TAS-20 when practicable, computing subscale scores, and exercising caution in interpreting TAS-20 EOT scores, particularly in psychoactive substance dependent-abusing inpatient samples.

Psychological implications of the Psychopathy Q-sort.

The construct validity of the Psychopathy Q-sort (PQS; Reise & Oliver, 1994) was investigated by correlating it with an array of observer-based and self-report personality measures in a sample of 350 men and women assessed at the Institute of Personality Assessment and Research (IPAR). High discriminant validity of the PQS was indicated by a pattern of significant correlations with self-report measures of the Cluster B personality disorder scales of the Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; American Psychiatric Association, 1980) and the absence of correlations with scales belonging to Clusters A and C of the manual. The PQS also correlated positively with the California Psychological Inventory measures of social poise and assurance and correlated negatively with measures of normative control of impulse. Gender differences in psychopathy were explored using Adjective Check List (Gough & Heilbrun, 1983) ratings provided by IPAR staff observers and by 76 pairs of spouses.

Developmental status, gender, age, and self-reported decision-making influences on students' risky and preventive health behaviors.

This study used decision-making theory to analyze the developmental changes associated with children's and adolescents' health behavior. High school and elementary school children completed surveys concerning (1) the extent to which they engage in a variety of preventive and risky health behaviors, and (2) influence sources used in decision making concerning the enactment of these behaviors. Multiple regression analysis revealed that the sources of influence children and adolescents report considering in making health-related decisions change developmentally and as a function of gender. Moreover, within and across age, children's sources of influence with respect to health decision making are dependent on the health domains being considered. The findings are discussed in relation to decision-making theory and the implications for the content and timing of health education initiatives for adolescents.

Personality dimensions and measures potentially relevant to health: A focus on hostility.

Although it is clear that chronic, negative socioemotional patterns are associated with poor health and premature death, various overlapping concepts and measures are currently used, making research planning difficult and limiting attempts at theoretical development. This article reviews current issues and reports a study of 36 theoretically interesting and commonly-used personality scales that were administered to 454 undergraduates. The scales included the Cook-Medley scale, Buss-Durkee Hostility Inventory, the Spielberger scales, Eysenck Personality Questionnaire, hardiness/alienation scales, NEO Personality Inventory, locus of control, Life Orientation Test, Jenkins Activity Survey, and the Beck Depression Inventory. While research and theory refine the best concepts and measures, studies predicting health from chronic negative patterns may want to include at least four measures: (a) aggressive overt hostility, (b) alienated bitterness, (c) introversion, and(d) anxiety/depression. A measure of conscientiousness is also useful.

Modeling Qualitative Variation Within Latent Trait Dimensions: Application of Mixed-Measurement to Personality Assessment.

The mixed-measurement IRT model and the traditional factor analytic model are two fundamentally different ways of representing the structure underlying an item response matrix. To contrast these approaches, the parameters of full-information item factor models of varying dimensionality and mixed-measurement models of varying numbers of latent classes were estimated in a sample of 1,000 responses to a 14-item measure of Positive Interpersonal Engagement. Findings indicated that either a two latent factor or a two latent class mixed-model provided the most appropriate representation of the data. We also found that the factor models were, in general, more parsimonious than the mixed-measurement models. Nevertheless, we argue that deciding between a dimensional and a mixed- measurement representation of item response heterogeneity should not rest solely on statistical criteria. In the discussion, we suggest some research contexts in which the mixed- measurement model may be conceptually more appropriate than the traditional factor model.

Development of a California Q-set indicator of primary psychopathy.

Seven judges described the personality characteristics of the primary psychopath by sorting the 100 items of the California Q-set in a forced-normal distribution. Item scores resulting from these sorts were aggregated across judges to form a Psychopathy Prototype. The reliability of the seven-judge aggregate prototype was .90. To examine the reliability of scores derived from the prototype, the personalities of 65 target subjects were described by two peer judges using the California Q-set (Block, 1961). For each subject, Psychopathy Prototype scores were derived by correlating each judge's Q-sort profile with the Psychopathy Prototype. Findings indicated that the peer-generated psychopathy scores correlated r = .61 between judges. Using the Spearman-Brown formula, Psychopathy Prototype scores have reliabilities of .75, .82, and .86 when aggregated over two, three, and four peer judges, respectively. To further explore properties of the measure, the Psychopathy Prototype was compared with independently developed California Q-set prototypes describing the narcissist and the female hysteric. The results revealed some interesting contrasts among these concepts and serve to support our contention that the Psychopathy Prototype has utility in regard to distinguishing between pathologies with overlapping features.

Genetic and environmental influences on item response pattern scalability.

Numerous studies have examined how genetic and environmental factors determine individual differences on multi-item personality scales. Few studies, however, have examined how genes and the environment influence the route by which individuals obtain their scores on these scales. Specifically, on a multi-item test, dozens of item response patterns result in equivalent total scores, though some response patterns are more likely to be observed than others. For many scales it may be of interest to determine the genetic and environmental influences on the item response patterns, as well as the sum of the item responses. We discuss a latent trait measure of item response pattern scalability, called Zl (Levine and Drasgow, 1982), and investigate the properties of this index from a behavioral genetics perspective. Using a large sample of identical and fraternal twins from the Minnesota Twin Registry (Lykken et al., 1990), item response pattern scalability is shown to be moderately heritable. On the four scales of the Multidimensional Personality Questionnaire (Tellegen, 1982) that were investigated, approximately 20% of the variation in scalability was due to genetic diversity between subjects of our sample. Follow-up analyses, using a factor-analytically based, genotype-environment model of item response behavior, indicated that specific genetic and environmental factors play a substantial role in determining item response pattern variation.