ABSTRACT
Bots are automated software programs that pose an ongoing threat to psychological research by invading online research studies and their increasing sophistication over time. Despite this growing concern, research in this area has been limited to bot detection in existing data sets following an unexpected encounter with bots. The present three-condition, quasi-experimental study aimed to address this gap in the literature by examining the efficacy of three types of bot screening tools across three incentive conditions ($0, $1, and $5). Data were collected from 444 respondents via Twitter advertisements between July and September 2021. The efficacy of five task-based (i.e., anagrams, visual search), question-based (i.e., attention checks, ReCAPTCHA), and data-based (i.e., consistency, metadata) tools was examined with Bonferroni-adjusted univariate and multivariate logistic regression analyses. In general, study results suggest that bot screening tools function similarly for participants recruited across incentive conditions. Moreover, the present analyses revealed heterogeneity in the efficacy of bot screening tool subtypes. Notably, the present results suggest that the least effective bot screening tools were among the most commonly used tools in existing literature (e.g., ReCAPTCHA). In sum, the study findings revealed highly effective and highly ineffective bot screening tools. Study design and data integrity recommendations for researchers are provided. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
ABSTRACT
Intraindividual change over time is commonly used to estimate treatment effectiveness. However, patients may not respond similarly to a scale after treatment, rendering pre-post change an unreliable metric. The current objective was to investigate longitudinal measurement invariance of the Patient Health Questionnaire-9 and Generalized Anxiety Disorder Scale-7 among 4,323 patients completing a partial hospital program. We used confirmatory factor analysis to determine (1) factor structure at pretreatment and posttreatment and (2) longitudinal invariance, accounting for dependent observations, using both classical and approximate measurement invariance approaches. Results indicated a two-factor solution for both scales. Longitudinal invariance was not established for either scale, thus, using raw score differences from the Patient Health Questionnaire-9 and Generalized Anxiety Disorder Scale-7 for measuring symptom change over time may be problematic. The most longitudinally consistent items captured somatic as opposed to affective/cognitive symptoms. We discuss the potential use of these measures for diagnostic screening and between-group comparisons and suggest alternative ways to monitor client progress over time. Limitations included a majority White sample and uniqueness of a partial hospital setting.
Subject(s)
Mass Screening , Patient Health Questionnaire , Humans , Factor Analysis, StatisticalABSTRACT
There are significant challenges in addressing the mental health needs of college students. The current study tested an acceptance and commitment therapy (ACT), web-based self-help program to treat a broad range of psychological problems students struggle with. A sample of 79 college students was randomized to web-based ACT or a waitlist condition, with assessments at baseline and posttreatment. Results indicated adequate acceptability and program engagement for the ACT website. Relative to waitlist, participants receiving ACT improved on overall distress, general anxiety, social anxiety, depression, academic concerns, and positive mental health. There were no between-group effects on eating concerns, alcohol use, or hostility, or on some key ACT process of change measures. ACT participants improved more on mindful acceptance and obstruction to valued living, both of which mediated treatment outcomes. Results are discussed in the context of lessons learned with the website prototype, and areas for further research are presented.
