Rhetoric of psychological measurement theory and practice.

Metascience scholars have long been concerned with tracking the use of rhetorical language in scientific discourse, oftentimes to analyze the legitimacy and validity of scientific claim-making. Psychology, however, has only recently become the explicit target of such metascientific scholarship, much of which has been in response to the recent crises surrounding replicability of quantitative research findings and questionable research practices. The focus of this paper is on the rhetoric of psychological measurement and validity scholarship, in both the theoretical and methodological and empirical literatures. We examine various discourse practices in published psychological measurement and validity literature, including: (a) clear instances of rhetoric (i.e., persuasion or performance); (b) common or rote expressions and tropes (e.g., perfunctory claims or declarations); (c) metaphors and other "literary" styles; and (d) ambiguous, confusing, or unjustifiable claims. The methodological approach we use is informed by a combination of conceptual analysis and exploratory grounded theory, the latter of which we used to identify relevant themes within the published psychological discourse. Examples of both constructive and useful or misleading and potentially harmful discourse practices will be given. Our objectives are both to contribute to the critical methodological literature on psychological measurement and connect metascience in psychology to broader interdisciplinary examinations of science discourse.

Measuring skill-based health literacy in chronic airway disease patients: the development and psychometric evaluation of the Vancouver airways health literacy tool (VAHLT).

PURPOSE: This article describes the development of the Vancouver airways health literacy tool (VAHLT), a novel measure of skill-based health literacy specific to chronic airway diseases (CADs). Across several phases, psychometric characteristics of the VAHLT were examined and used to guide its development. METHODS: An initial pool of 46 items was developed using input from patients, clinicians, researchers, and policy-makers. An initial patient sample (N = 532) was evaluated and used to inform item revisions. A revised 44-item pool was then evaluated using a second sample, the results of which aided in the selection of a final set of 30 items. The finalized 30-item VAHLT was then psychometrically evaluated using the second sample (N = 318). An item response theory approach was utilized to evaluate the VAHLT by assessing model fit, item parameter estimates, test and item information curves, and item characteristic curves. Reliability was assessed using ordinal coefficient alpha. We additionally assessed differential item functioning between asthma and COPD diagnoses. RESULTS: The VAHLT demonstrated a unidimensional structure and reasonably discriminated patients in the lower range of health literacy estimates. The tool demonstrated strong reliability (α = .920). Two of the 30 items were found to exhibit non-negligible differential item functioning. CONCLUSIONS: This study presents compelling evidence of validity in several areas for the VAHLT, including content and structural validity. Further external validation studies are needed and forthcoming. Overall, this work represents a strong first step towards a novel, skill-based, and disease-specific measure of CAD-related health literacy.

Pregnancy-Specific Anxiety Tool (PSAT): Instrument Development and Psychometric Evaluation.

Objective: Pregnancy-specific anxiety (PSA) is a distinct construct from general anxiety and depression. The purpose of this study was to develop, evaluate, and validate the Pregnancy-Specific Anxiety Tool (PSAT), to measure PSA and its severity.Methods: The study was carried out in 2 stages. Stage 1 involved item development and content and face validation. Stage 2 included psychometric evaluation to examine item distributions and correlational structure, dimensionality, internal consistency reliability, stability, and construct, convergent, and criterion validity, using 2 independent samples (initial sample N = 494, May-October 2018; validation sample N = 325, July 2019-May 2020).Results: Eighty-two items were evaluated for face validity and 41 items were considered in stage 2 based on feedback from participants and experts. Model fit from exploratory factor analysis and patterns of item-factor loadings suggested a 6-factor model with 33 items. The 6 factors included items pertaining to health and well-being of the baby, labor and the pregnant person's well-being, postpartum, support, career and finance, and indicators of severity. Confirmatory factor analysis carried out using the initial sample showed good fit with the validation sample. The area under the curve (AUC) for the diagnosis of adjustment disorders (AD) was 0.73 (95% CI, 0.67-0.79), and for AD/any anxiety disorders, the AUC was 0.80 (95% CI, 0.75-0.85).Conclusions: The PSAT can be useful for screening and monitoring of PSA, and pregnant people with scores higher than 10 should be considered for further assessment.

A risk perception model to promote smoking cessation among adult current smokers: Application of a community-based smoking cessation intervention.

INTRODUCTION: This study aims to assess the changes in cigarette consumption CO levels, and importance and self-confidence levels of quitting smoking in adult smokers after participation in a 6-month community-based smoking cessation program. METHODS: Participants were enrolled in the intervention (counseling group) or noncounseling group after completing a baseline assessment and receiving educational materials. A final assessment was completed at 6 months and at 8 months. Mixed factorial analyses of variance (ANOVAs) with 2 (condition: counseling vs non-counseling) × 2 (spoken language: Chinese vs English) × 2 (time: initial measurement vs last measurement) designs, were conducted for each of the four outcome measures. RESULTS: Seventy smokers (50 males, 20 females; mean age 34 years, SD=9.68) from the Mandarin- and Cantonese-speaking (n=30) and English-speaking (n=40) communities in Vancouver, Canada were enrolled. Cigarette consumption: Both Chinese- and English-speaking participants significantly decreased their cigarette consumption at near equal rates and English-speaking participants smoked significantly more cigarettes than the Chinese-speaking participants did across both the initial and last measurement time points. No significant interactions or main effects related to the study condition variable were observed. CO levels: Average levels of CO also decreased near equally for the English- and Chinese-speaking groups. No significant interactions or main effects related to the study condition variable were observed. Importance ratings: Chinese-speakers in the non-counseling group showed no increase in importance ratings, whereas Chinese-speakers in the counseling group saw an average increase of 0.73 (SD=1.10). For the English-speaking group, importance ratings remained stable over time with no significant changes. Self-confidence ratings: A larger average rating increase for the counseling group compared to the non-counseling group was observed for the study condition and time variables. On average, confidence ratings increased significantly for the whole sample. CONCLUSIONS: Our study showed that participation in a community-based smoking cessation intervention study, regardless of study group, resulted in significant decreases over time in self-reported cigarettes smoked per day and measured CO levels; as well as a significant effect on importance ratings depending on the language group, and a significant increase in confidence ratings overall.

Construct Validity of the Comprehensive Assessment of Psychopathic Personality (CAPP): Examining the Internal Structure and Generalizability of CAPP Self-Ratings Across Gender and Ethnicity.

The Comprehensive Assessment of Psychopathic Personality (CAPP) is a recently developed conceptual model of psychopathy designed to index the disorder across 33 personality traits. Although recent research has evidenced support for the CAPP model with respect to the convergent, criterion, and predictive validity of CAPP instruments, little work has examined the optimal internal structure and generalizability of the model and associated measures. The present study sought to elaborate on the construct validity and psychometric properties of the CAPP Lexical Self Rating Scale and determine the utility of the instrument across men and women, and individuals of Caucasian and East Asian descent. Within a large sample of self-identified offenders (N = 1,414), we found strong support for a three-factor model comprising domains reflecting interpersonal dominance, behavioral disinhibition, and deficient emotional attachment. Analyses examining the generalizability of the model provide further insight into its applicability for diverse populations.

Primary Study Quality in Psychological Meta-Analyses: An Empirical Assessment of Recent Practice.

As meta-analytic research has come to occupy a sizeable contingent of published work in the psychological sciences, clarity in the reporting of such work is crucial to its interpretability and reproducibility. This is especially true regarding the assessment of primary study quality, as notions of study quality can vary across research domains. The present study examines the general state of reporting practices related to primary study quality in a sample of 382 published psychological meta-analyses, as well as the reporting decisions and motivations of the authors that published them. Our findings suggest adherence to reporting standards has remained poor for assessments of primary study quality and that the discipline remains inconsistent in its reporting practices generally. We discuss several potential reasons for the poor adherence to reporting standards in our sample, including whether quality assessments are being conducted in the first place, whether standards are well-known within the discipline, and the potential conflation of assessing primary study quality with other facets of conducting a meta-analysis. The implications of suboptimal reporting practices related to primary study quality are discussed.