RESUMEN
Medicine regulators need to judge whether a drug's favorable effects outweigh its unfavorable effects based on a dossier submitted by an applicant, such as a pharmaceutical company. Because scientific knowledge is inherently uncertain, regulators also need to judge the credibility of these effects by identifying and evaluating uncertainties. We performed an ethnographic study of assessment procedures at the Dutch Medicines Evaluation Board (MEB) and describe how regulators evaluate the credibility of an applicant's claims about the benefits and risks of a drug in practice. Our analysis shows that regulators use an investigative approach, which illustrates the effort required to identify uncertainties. Moreover, we show that regulators' expectations about the presentation, the design, and the results of studies can shape how they perceive a medicine's dossier. We highlight the importance of regulatory experience and expertise in the identification and evaluation of uncertainties. In light of our observations, we provide two recommendations to reduce avoidable uncertainty: less reliance on evidence generated by the applicant; and better communication about, and enforcement of, regulatory frameworks toward drug developers.
RESUMEN
OBJECTIVES: To quantify the strength of statistical evidence of randomized controlled trials (RCTs) for novel cancer drugs approved by the Food and Drug Administration in the last 2 decades. STUDY DESIGN AND SETTING: We used data on overall survival (OS), progression-free survival, and tumor response for novel cancer drugs approved for the first time by the Food and Drug Administration between January 2000 and December 2020. We assessed strength of statistical evidence by calculating Bayes factors (BFs) for all available endpoints, and we pooled evidence using Bayesian fixed-effect meta-analysis for indications approved based on 2 RCTs. Strength of statistical evidence was compared among endpoints, approval pathways, lines of treatment, and types of cancer. RESULTS: We analysed the available data from 82 RCTs corresponding to 68 indications supported by a single RCT and 7 indications supported by 2 RCTs. Median strength of statistical evidence was ambiguous for OS (BF = 1.9; interquartile range [IQR] 0.5-14.5), and strong for progression-free survival (BF = 24,767.8; IQR 109.0-7.3 × 106) and tumor response (BF = 113.9; IQR 3.0-547,100). Overall, 44 indications (58.7%) were approved without clear statistical evidence for OS improvements and 7 indications (9.3%) were approved without statistical evidence for improvements on any endpoint. Strength of statistical evidence was lower for accelerated approval compared to nonaccelerated approval across all 3 endpoints. No meaningful differences were observed for line of treatment and cancer type. This analysis is limited to statistical evidence. We did not consider nonstatistical factors (eg, risk of bias, quality of the evidence). CONCLUSION: BFs offer novel insights into the strength of statistical evidence underlying cancer drug approvals. Most novel cancer drugs lack strong statistical evidence that they improve OS, and a few lack statistical evidence for efficacy altogether. These cases require a transparent and clear explanation. When evidence is ambiguous, additional postmarketing trials could reduce uncertainty.
RESUMEN
BACKGROUND: Dendritic cell immunotherapy has proven to be safe and induces an immune response in humans. We aimed to establish the efficacy of dendritic cells loaded with allogeneic tumour cell lysate (MesoPher, Amphera BV, 's-Hertogenbosch, Netherlands) as maintenance therapy in patients with pleural mesothelioma. METHODS: In this open-label, randomised, phase 2/3 study, patients with histologically confirmed unresectable pleural mesothelioma, aged 18 years or older, with an Eastern Cooperative Oncology Group performance status score of 0-1, and non-progressing disease after four to six cycles of standard chemotherapy (with pemetrexed 500 mg/m2 plus platinum [cisplatin 75 mg/m2 or carboplatin area under the curve of 5]) were recruited from four centres in Belgium, France, and The Netherlands. Participants were randomly assigned (1:1), using block randomisation (block size of 4), stratified by centre and histology (epithelioid vs other), to MesoPher treatment plus best supportive care or best supportive care alone. Patients received up to a maximum of five MesoPher infusions, with treatment administered on days 1, 15, and 29, and weeks 18 and 30. At each timepoint, participants received an injection of 25 × 106 dendritic cells (two-thirds of the dendritic cells were administered intravenously and a third were injected intradermally). Best supportive care was per local institutional standards. The primary endpoint was overall survival, assessed in all participants randomly assigned to treatment (full analysis set) and safety assessed in all randomly assigned participants, and who underwent leukapheresis if they were in the MesoPher group. This study is registered with ClinicalTrials.gov, NCT03610360, and is closed for accrual. FINDINGS: Between June 21, 2018, and June 10, 2021, 176 patients were screened and randomly assigned to the MesoPher group (n=88) or best supportive care alone group (n=88). One participant in the MesoPher group did not undergo leukapheresis. Mean age was 68 years (SD 8), 149 (85%) of 176 were male, 27 (15%) were female, 173 (98%) were White, two were Asian (1%), and one (1%) was other race. As of data cutoff (June 24, 2023), after a median follow up of 15·1 months (IQR 9·5-22·4), median overall survival was 16·8 months (95% CI 12·4-20·3; 61 [69%] of 88 died) in the MesoPher group and 18·3 months (14·3-21·9; 59 [67%] of 88 died) in the best supportive care group (hazard ratio 1·10 [95% CI 0·77-1·57]; log-rank p=0·62). The most common grade 3-4 treatment-emergent adverse events were chest pain (three [3%] of 87 in the MesoPher group vs two [2%] of 88 in the best supportive care group), dyspnoea (none vs two [2%]), anaemia (two [2%] vs none), nausea (none vs two [2%]), and pneumonia (none vs two [2%]). No deaths due to treatment-emergent adverse events were recorded. Treatment-related adverse events consisted of infusion-related reactions (fever, chills, and fatigue), which occurred in 64 (74%) of 87 patients in the MesoPher group, and injection-site reactions (itch, erythema, and induration), which occurred in 73 (84%) patients, and all were grade 1-2 in severity. No deaths were determined to be treatment related. INTERPRETATION: MesoPher did not show improvement in overall survival in patients with pleural mesothelioma. Immune checkpoint therapy is now standard of care in pleural mesothelioma. Further randomised studies are needed of combinations of MesoPher and immune checkpoint therapy, which might increase efficacy without adding major toxicities. FUNDING: Amphera BV and EU HORIZON.
Asunto(s)
Células Dendríticas , Neoplasias Pleurales , Humanos , Femenino , Masculino , Células Dendríticas/trasplante , Células Dendríticas/inmunología , Anciano , Persona de Mediana Edad , Neoplasias Pleurales/terapia , Neoplasias Pleurales/patología , Neoplasias Pleurales/mortalidad , Neoplasias Pleurales/tratamiento farmacológico , Neoplasias Pleurales/inmunología , Mesotelioma/terapia , Mesotelioma/tratamiento farmacológico , Mesotelioma/patología , Mesotelioma/mortalidad , Mesotelioma/inmunología , Protocolos de Quimioterapia Combinada Antineoplásica/uso terapéutico , Protocolos de Quimioterapia Combinada Antineoplásica/efectos adversos , Protocolos de Quimioterapia Combinada Antineoplásica/administración & dosificación , Mesotelioma Maligno/terapia , Mesotelioma Maligno/patología , Mesotelioma Maligno/tratamiento farmacológico , Quimioterapia de Mantención , Cisplatino/administración & dosificación , Carboplatino/administración & dosificación , Pemetrexed/administración & dosificaciónRESUMEN
Several new models based on item response theory have recently been suggested to analyse intensive longitudinal data. One of these new models is the time-varying dynamic partial credit model (TV-DPCM; Castro-Alvarez et al., Multivariate Behavioral Research, 2023, 1), which is a combination of the partial credit model and the time-varying autoregressive model. The model allows the study of the psychometric properties of the items and the modelling of nonlinear trends at the latent state level. However, there is a severe lack of tools to assess the fit of the TV-DPCM. In this paper, we propose and develop several test statistics and discrepancy measures based on the posterior predictive model checking (PPMC) method (PPMC; Rubin, The Annals of Statistics, 1984, 12, 1151) to assess the fit of the TV-DPCM. Simulated and empirical data are used to study the performance of and illustrate the effectiveness of the PPMC method.
RESUMEN
The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are still great challenges to overcome as, in many cases, collected data are more complex than the available models are able to handle. For example, most methods assume that the variables in the time series are measured on an interval scale, which is not the case when Likert-scale items were used. Ignoring the scale of the variables can be problematic and bias the results. Additionally, most methods also assume that the time series are stationary, which is rarely the case. To tackle these disadvantages, we propose a model that combines the partial credit model (PCM) of the item response theory framework and the time-varying autoregressive model (TV-AR), which is a popular model used to study psychological dynamics. The proposed model is referred to as the time-varying dynamic partial credit model (TV-DPCM), which allows to appropriately analyze multivariate polytomous data and nonstationary time series. We test the performance and accuracy of the TV-DPCM in a simulation study. Lastly, by means of an example, we show how to fit the model to empirical data and interpret the results.
Asunto(s)
Modelos Estadísticos , Factores de Tiempo , Simulación por Computador , Recolección de DatosRESUMEN
Robust scientific evidence shows that human performance predictions are more valid when information is combined mechanically (with a decision rule) rather than holistically (in the decision-maker's mind). Yet, information is often combined holistically in practice. One reason is that decision makers lack the knowledge of evidence-based decision making. In a performance prediction task, we tested whether watching an educational video on evidence-based decision making increased decision-makers' use of a decision rule and their prediction accuracy immediately after the manipulation and a month later. Furthermore, we manipulated whether participants earned incentives for accurate predictions. Existing research showed that incentives decrease decision-rule use and prediction accuracy. We hypothesized that this is the case for decision makers who did not receive educational information about evidence-based decision making, but that incentives increase decision-rule use and prediction accuracy for participants who received educational information. Our results showed that educational information increased decision-rule use. This resulted in increased prediction accuracy, but only immediately after receiving the educational information. In contrast to the existing literature, incentives slightly increased decision-rule use. We did not find evidence that this effect was larger for educated participants. Providing decision makers with educational information may be effective to increase decision-rule use in practice. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Asunto(s)
Toma de Decisiones , Motivación , HumanosRESUMEN
Scouts of soccer clubs are often the first to identify talented players. However, there is a lack of research on how these scouts assess and predict overall soccer performance. Therefore, we conducted a large-scaled study to examine the process of talent identification among 125 soccer scouts. Through an online self-report questionnaire, scouts were asked about (1) the players' age at which they can predict players' soccer performance, (2) the attributes they consider relevant, and (3) the extent to which they predict performance in a structured manner. The most important results are as follows. First, scouts who observed 12-year-old and younger players perceived they could predict at older ages (13.6 years old, on average) whether a player has the potential to become a professional soccer player. This suggests that scouts are aware of the idea that early indicators of later performance are often lacking, yet do advise on selection of players at younger ages. Second, when identifying talented players, scouts considered more easily observable attributes, such as technical attributes. However, scouts described these often in a broad sense rather than in terms of specific predictors of future performance. Finally, scouts reported that they assess attributes of players in a structured manner. Yet, they ultimately based their prediction (i.e. final score) on an intuitive integration of different performance attributes, which is a suboptimal strategy according to existing literature. Taken together, these outcomes provide specific clues to improve the reliability and validity of the scouting process. HighlightsBased on a large sample of soccer scouts, we examine three issues that are important in the process of identifying talented soccer players: The age at which good performance predictions can be made, which attributes are relevant predictors, and how performance predictions are formed.Scouts who observe players in young age cohorts believe that the age at which they can predict performance is older than the players they typically scout, suggesting that they are aware that early indicators of performance are often lacking.Technical performance attributes are considered as most important performance predictors by scouts, but these are often described in a broad - rather than specific - sense.Scouts indicate that they predict performance in a structured manner, but form their overall performance prediction on an intuitive integration of different performance attributes, which can be a suboptimal approach.
Asunto(s)
Rendimiento Atlético , Fútbol , Adolescente , Aptitud , Niño , Humanos , Reproducibilidad de los ResultadosRESUMEN
Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome these drawbacks is to consider structural equation models (state-trait SEMs) for longitudinal data that represent traits and states as latent variables. Most of these models are encompassed in the latent state-trait (LST) theory. These state-trait SEMs can be problematic when the number of measurement occasions increases. As they require the data to be in wide format, these models quickly become overparameterized and lead to nonconvergence issues. For these reasons, multilevel versions of state-trait SEMs have been proposed, which require the data in long format. To study how suitable state-trait SEMs are for intensive longitudinal data, we carried out a simulation study. We compared the traditional single level to the multilevel version of three state-trait SEMs. The selected models were the multistate-singletrait (MSST) model, the common and unique trait-state (CUTS) model, and the trait-state-occasion (TSO) model. Furthermore, we also included an empirical application. Our results indicated that the TSO model performed best in both the simulated and the empirical data. To conclude, we highlight the usefulness of state-trait SEMs to study the psychometric properties of the questionnaires used in intensive longitudinal data. Yet, these models still have multiple limitations, some of which might be overcome by extending them to more general frameworks. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Asunto(s)
Modelos Teóricos , Humanos , Análisis de Clases Latentes , Análisis Multinivel , Psicometría , Encuestas y CuestionariosRESUMEN
PURPOSE: In Mokken scaling, the Crit index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To make the formulation of the Crit index explicit and accessible, and to investigate its distribution under various measurement conditions. METHODS: We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the Crit coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health. RESULTS: We found that the false positive rates of Crit were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples Crit lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the Crit index may fail to detect assumption violations. DISCUSSION: Even in large samples, the Crit coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision.
Asunto(s)
Calidad de Vida , Proyectos de Investigación , Simulación por Computador , Humanos , Salud Mental , Calidad de Vida/psicología , Encuestas y CuestionariosRESUMEN
BACKGROUND: In first episode psychosis (FEP) baseline negative symptoms (BNS) and relapse both predict less favorable functional outcome. Relapse-prevention is one of the most important goals of treatment. Apart from discontinuation of antipsychotics, natural causes of relapse are unexplained. We hypothesized that BNS, apart from predicting worse functional outcome, might also increase relapse risk. METHODS: We performed a post-hoc analysis of 7-year follow-up data of a FEP cohort (n = 103) involved in a dose-reduction/discontinuation (DR) vs. maintenance treatment (MT) trial. We examined: 1) what predicted relapse, 2) what predicted functional outcome, and 3) if BNS predicted relapse, whether MT reduced relapse rates compared to DR. After remission patients were randomly assigned to DR or MT for 18 months. Thereafter, treatment was uncontrolled. OUTCOMES: BNS and duration of untreated psychosis (DUP) predicted relapse. Number of relapses, BNS, and treatment strategy predicted functional outcome. BNS was the strongest predictor of relapse, while number of relapses was the strongest predictor of functional outcome above BNS and treatment strategy. Overall and within MT, but not within DR, more severe BNS predicted significantly higher relapse rates. Treatment strategies did not make a difference in relapse rates, regardless of BNS severity. INTERPRETATION: BNS not only predicted worse functional outcome, but also relapses during follow-up. Since current low dose maintenance treatment strategies did not prevent relapse proneness in patients with more severe BNS, resources should be deployed to find optimal treatment strategies for this particular group of patients.
Asunto(s)
Antipsicóticos , Trastornos Psicóticos , Antipsicóticos/uso terapéutico , Humanos , Trastornos Psicóticos/tratamiento farmacológico , Recurrencia , Prevención Secundaria , Resultado del TratamientoRESUMEN
Predicting performance in soccer games has been a major focus within talent identification and development. Past research has mainly used performance levels, such as elite vs. non-elite players, as the performance to predict (i.e. the criterion). Moreover, these studies have mainly focused on isolated performance attributes as predictors of soccer performance levels. However, there has been an increasing interest in finer grained criterion measures of soccer performance, as well as representative assessments at the level of performance predictors. In this study, we first determined the degree to which 7-vs-7 small-sided games can be considered as representative of 11-vs-11 games. Second, we assessed the validity of individual players' small-sided game performance in predicting their 11-vs-11 game performance on a continuous scale. Moreover, we explored the predictive validity for 11-vs-11 game performance of several physiological and motor tests in isolation. Sixty-three elite youth players of a professional soccer academy participated in 11 to 17 small-sided games and six 11-vs-11 soccer games. In-game performance indicators were assessed through notational analysis and combined into an overall offensive and defensive performance measure, based on their relationship with game success. Physiological and motor abilities were assessed using a sprint, endurance, and agility test. Results showed that the small-sided games were faster paced, but representative of 11-vs-11 games, with the exception of aerial duels. Furthermore, individual small-sided game performance yielded moderate predictive validities with 11-vs-11 game performance. In contrast, the physiological and motor tests yielded small to trivial relations with game performance. Altogether, this study provides novel insights into the application of representative soccer assessments and the use of continuous criterion measures of soccer performance.
Asunto(s)
Rendimiento Atlético/estadística & datos numéricos , Fútbol/fisiología , Adolescente , Humanos , Masculino , Modelos Estadísticos , Actividad Motora , Reproducibilidad de los ResultadosRESUMEN
Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thumb. Using simulated data, they concluded that items with low scalability had some influence on the reliability of test scores, person ordering and selection, and criterion-related validity estimates. Removing the misfitting items from the scale had, in general, a small effect on the outcomes. Although important outcome variables were fairly robust against scale violations in some conditions, authors conclude that researchers should not rely exclusively on algorithms allowing automatic selection of items. In particular, content validity must be taken into account to build sensible psychometric instruments.
RESUMEN
In the past decades, much research has examined the negative effects of stressors on the performance of athletes. However, according to evolutionary biology, organisms may exhibit growth under stress, a phenomenon called antifragility. For both coaches and their athletes, a key question is how to design training conditions to help athletes develop the kinds of physical, physiological, and behavioral adaptations underlying antifragility. An answer to this important question requires a better understanding of how individual athletes respond to stress or loads in the context of relevant sports tasks. In order to contribute to such understanding, the present study leverages a theoretical and methodological approach to generate individualized load-response profiles in the context of a climbing task. Climbers (n = 37) were asked to complete different bouldering (climbing) routes with increasing loading (i.e. difficulty). We quantified the behavioral responses of each individual athlete by mathematically combining two measures obtained for each route: (a) maximal performance (i.e. the percentage of the route that was completed) and (b) number of attempts required to achieve maximal performance. We mapped this composite response variable as a function of route difficulty. This procedure resulted in load-response curves that captured each athlete's adaptability to stress, termed phenotypic plasticity (PP), specifically operationalized as the area under the generated curves. The results indicate individual load-response profiles (and by extension PP) for athletes who perform at similar maximum levels. We discuss how these profiles might be used by coaches to systematically select stress loads that may be ideally featured in performance training.
RESUMEN
BACKGROUND: In first episode psychosis (FEP) baseline negative symptoms (BNS) and relapse both predict less favorable functional outcome. Relapse-prevention is one of the most important goals of treatment. Apart from discontinuation of antipsychotics, natural causes of relapse are unexplained. We hypothesized that BNS, apart from predicting worse functional outcome, might also increase relapse risk. METHODS: We performed a post-hoc analysis of 7-year follow-up data of a FEP cohort (n = 103) involved in a dose-reduction/discontinuation (DR) vs. maintenance treatment (MT) trial. We examined: 1) what predicted relapse, 2) what predicted functional outcome, and 3) if BNS predicted relapse, whether MT reduced relapse rates compared to DR. After remission patients were randomly assigned to DR or MT for 18 months. Thereafter, treatment was uncontrolled. OUTCOMES: BNS and duration of untreated psychosis (DUP) predicted relapse. Number of relapses, BNS, and treatment strategy predicted functional outcome. BNS was the strongest predictor of relapse, while number of relapses was the strongest predictor of functional outcome above BNS and treatment strategy. Overall and within MT, but not within DR, more severe BNS predicted significantly higher relapse rates. Treatment strategies did not make a difference in relapse rates, regardless of BNS severity. INTERPRETATION: BNS not only predicted worse functional outcome, but also relapses during follow-up. Since current low dose maintenance treatment strategies did not prevent relapse proneness in patients with more severe BNS, resources should be deployed to find optimal treatment strategies for this particular group of patients.
Asunto(s)
Antipsicóticos , Trastornos Psicóticos , Esquizofrenia , Antipsicóticos/uso terapéutico , Humanos , Trastornos Psicóticos/tratamiento farmacológico , Recurrencia , Esquizofrenia/tratamiento farmacológico , Prevención Secundaria , Resultado del TratamientoRESUMEN
OBJECTIVES: In this study, we examined the consequences of ignoring violations of assumptions underlying the use of sum scores in assessing attention problems (AP) and if psychometrically more refined models improve predictions of relevant outcomes in adulthood. METHODS: Tracking Adolescents' Individual Lives data were used. AP symptom properties were examined using the AP scale of the Child Behavior Checklist at age 11. Consequences of model violations were evaluated in relation to psychopathology, educational attainment, financial status, and ability to form relationships in adulthood. RESULTS: Results showed that symptoms differed with respect to information and difficulty. Moreover, evidence of multidimensionality was found, with two groups of items measuring sluggish cognitive tempo and attention deficit hyperactivity disorder symptoms. Item response theory analyses indicated that a bifactor model fitted these data better than other competing models. In terms of accuracy of predicting functional outcomes, sum scores were robust against violations of assumptions in some situations. Nevertheless, AP scores derived from the bifactor model showed some superiority over sum scores. CONCLUSION: These findings show that more accurate predictions of later-life difficulties can be made if one uses a more suitable psychometric model to assess AP severity in children. This has important implications for research and clinical practice.
Asunto(s)
Trastorno por Déficit de Atención con Hiperactividad/diagnóstico , Escala de Evaluación de la Conducta/normas , Trastornos de la Conducta Infantil/diagnóstico , Escalas de Valoración Psiquiátrica/normas , Psicometría/normas , Adolescente , Adulto , Niño , Femenino , Humanos , Estudios Longitudinales , Masculino , Modelos Estadísticos , Índice de Severidad de la Enfermedad , Adulto JovenRESUMEN
Talent identification research in soccer comprises the prediction of elite soccer performance. While many studies in this field have aimed to empirically relate performance characteristics to subsequent soccer success, a critical evaluation of the methodology of these studies has mostly been absent in the literature. In this position paper, we discuss advantages and limitations of the design, validity, and utility of current soccer talent identification research. Specifically, we draw on principles from selection psychology that can contribute to best practices in the context of making selection decisions across domains. Based on an extensive search of the soccer literature, we identify four methodological issues from this framework that are relevant for talent identification research, i.e. (1) the operationalization of criterion variables (the performance to be predicted) as performance levels; (2) the focus on isolated performance indicators as predictors of soccer performance; (3) the effects of range restriction on the predictive validity of predictors used in talent identification; and (4) the effect of the base rate on the utility of talent identification procedures. Based on these four issues, we highlight opportunities and challenges for future soccer talent identification studies that may contribute to developing evidence-based selection procedures. We suggest for future research to consider the use of individual soccer criterion measures, to adopt representative, high-fidelity predictors of soccer performance, and to take restriction of range and the base rate into account.
Asunto(s)
Aptitud , Rendimiento Atlético , Toma de Decisiones , Fútbol , HumanosRESUMEN
BACKGROUND: The Four-Dimensional Symptom Questionnaire (4DSQ) is a self-report questionnaire designed to measure distress, depression, anxiety, and somatization. Prior to computing scale scores from the item scores, the three highest response alternatives ('Regularly', 'Often', and 'Very often or constantly present') are usually collapsed into one category to reduce the influence of extreme responding on item- and scale scores. In this study, we evaluate the usefulness of this transformation for the distress scale based on a variety of criteria. METHODS: Specifically, by using the Graded Response Model, we investigated the effect of this transformation on model fit, local measurement precision, and various indicators of the scale's validity to get an indication on whether the current practice of recoding should be advocated or not. In particular, the effect on the convergent- (operationalized by the General Health Questionnaire and the Maastricht Questionnaire), divergent- (operationalized by the Neuroticism scale of the NEO-FFI), and predictive validity (operationalized as obtrusion with daily chores and activities, the Biographical Problem list and the Utrecht Burnout Scale) of the distress scale was investigated. RESULTS: Results indicate that recoding leads to (i) better model fit as indicated by lower mean probabilities of exact test statistics assessing item fit, (ii) small (<.02) losses in the sizes of various validity coefficients, and (iii) a decrease (DIFF (SE's) = .10-.25) in measurement precision for medium and high levels of distress. CONCLUSIONS: For clinical applications and applications in longitudinal research, the current practice of recoding should be avoided because recoding decreases measurement precision for medium and high levels of distress. It would be interesting to see whether this advice also holds for the three other domains of the 4DSQ.
Asunto(s)
Ansiedad/diagnóstico , Depresión/diagnóstico , Autoinforme , Trastornos Somatomorfos/diagnóstico , Encuestas y Cuestionarios/normas , Adulto , Trastornos de Ansiedad/diagnóstico , Trastorno Depresivo/diagnóstico , Femenino , Humanos , Masculino , Persona de Mediana Edad , Psicometría , Reproducibilidad de los Resultados , Proyectos de InvestigaciónRESUMEN
In decision-making, it is important not only to use the correct information but also to combine information in an optimal way. There are robust research findings that a mechanical combination of information for personnel and educational selection matches or outperforms a holistic combination of information. However, practitioners and policy makers seldom use mechanical combination for decision-making. One of the important conditions for scientific results to be used in practice and to be part of policy-making is that results are easily accessible. To increase the accessibility of mechanical judgment prediction procedures, we (1) explain in detail how mechanical combination procedures work, (2) provide examples to illustrate these procedures, and (3) discuss some limitations of mechanical decision-making.