Adjusted Residuals for Evaluating Conditional Independence in IRT Models for Multistage Adaptive Testing.

The key assumption of conditional independence of item responses given latent ability in item response theory (IRT) models is addressed for multistage adaptive testing (MST) designs. Routing decisions in MST designs can cause patterns in the data that are not accounted for by the IRT model. This phenomenon relates to quasi-independence in log-linear models for incomplete contingency tables and impacts certain types of statistical inference based on assumptions on observed and missing data. We demonstrate that generalized residuals for item pair frequencies under IRT models as discussed by Haberman and Sinharay (J Am Stat Assoc 108:1435-1444, 2013. https://doi.org/10.1080/01621459.2013.835660 ) are inappropriate for MST data without adjustments. The adjustments are dependent on the MST design, and can quickly become nontrivial as the complexity of the routing increases. However, the adjusted residuals are found to have satisfactory Type I errors in a simulation and illustrated by an application to real MST data from the Programme for International Student Assessment (PISA). Implications and suggestions for statistical inference with MST designs are discussed.

No COVID-19 in Patients With Sudden Sensorineural Hearing Loss (SSNHL).

BACKGROUND: Various case reports have described sudden sensorineural hearing loss (SSNHL) in patients with the 2019 novel coronavirus disease (COVID-19) caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Our aim was to determine the incidence of COVID-19 in patients with SSNHL. METHODS: All consecutive patients with audiometric confirmed SSNHL between November 2020 and March 2021 in a Dutch large inner city teaching hospital were included. All patients were tested for COVID-19 by polymerase-chain-reaction (PCR) and awaited the results in quarantine. RESULTS: Out of 25 patients, zero (0%) tested positive for COVID-19. Two patients had previously tested positive for COVID-19: at three and eight months prior to the onset of hearing loss. CONCLUSIONS: This is the largest series to date investigating COVID-19 in SSNHL patients. In this series there is no apparent relationship between SSNHL and COVID-19.

Multidimensional Test Assembly Using Mixed-Integer Linear Programming: An Application of Kullback-Leibler Information.

Many educational testing programs require different test forms with minimal or no item overlap. At the same time, the test forms should be parallel in terms of their statistical and content-related properties. A well-established method to assemble parallel test forms is to apply combinatorial optimization using mixed-integer linear programming (MILP). Using this approach, in the unidimensional case, Fisher information (FI) is commonly used as the statistical target to obtain parallelism. In the multidimensional case, however, FI is a multidimensional matrix, which complicates its use as a statistical target. Previous research addressing this problem focused on item selection criteria for multidimensional computerized adaptive testing (MCAT). Yet these selection criteria are not directly transferable to the assembly of linear parallel test forms. To bridge this gap the authors derive different statistical targets, based on either FI or the Kullback-Leibler (KL) divergence, that can be applied in MILP models to assemble multidimensional parallel test forms. Using simulated item pools and an item pool based on empirical items, the proposed statistical targets are compared and evaluated. Promising results with respect to the KL-based statistical targets are presented and discussed.

Fetal fraction evaluation in non-invasive prenatal screening (NIPS).

An important factor in quality control of non-invasive prenatal screening (NIPS) or testing (NIPT) is a sufficient percentage of fetal DNA to avoid false-negative results. Here we evaluate 14,379 shallow whole-genome sequenced diagnostic NIPS samples, as well as negative controls, for both technical and biological factors that can influence fetal fraction and its assessment. Technically, bioinformatics analyses can have a profound impact on fetal fraction determination. We found best performance for fetal fraction determination with the Y chromosome based tool DEFRAG for male fetuses and the count based tool SeqFF for female fetuses. Biologically, gestational age of up to 21 weeks and maternal age had no influence on fetal fraction, while an increase in weight and BMI had a negative influence on fetal fraction. While a trend was observed, no statistically significant difference in fetal fraction was found between trisomy and normal samples. Overall, these results confirm the influence of biological factors and give insight into technical factors that can affect fetal fractions in NIPS.

Results of next-generation sequencing gene panel diagnostics including copy-number variation analysis in 810 patients suspected of heritable thoracic aortic disorders.

Simultaneous analysis of multiple genes using next-generation sequencing (NGS) technology has become widely available. Copy-number variations (CNVs) in disease-associated genes have emerged as a cause for several hereditary disorders. CNVs are, however, not routinely detected using NGS analysis. The aim of this study was to assess the diagnostic yield and the prevalence of CNVs using our panel of Hereditary Thoracic Aortic Disease (H-TAD)-associated genes. Eight hundred ten patients suspected of H-TAD were analyzed by targeted NGS analysis of 21 H-TAD associated genes. In addition, the eXome hidden Markov model (XHMM; an algorithm to identify CNVs in targeted NGS data) was used to detect CNVs in these genes. A pathogenic or likely pathogenic variant was found in 66 of 810 patients (8.1%). Of these 66 pathogenic or likely pathogenic variants, six (9.1%) were CNVs not detectable by routine NGS analysis. These CNVs were four intragenic (multi-)exon deletions in MYLK, TGFB2, SMAD3, and PRKG1, respectively. In addition, a large duplication including NOTCH1 and a large deletion encompassing SCARF2 were detected. As confirmed by additional analyses, both CNVs indicated larger chromosomal abnormalities, which could explain the phenotype in both patients. Given the clinical relevance of the identification of a genetic cause, CNV analysis using a method such as XHMM should be incorporated into the clinical diagnostic care for H-TAD patients.

A Generalized Speed-Accuracy Response Model for Dichotomous Items.

We propose a generalization of the speed-accuracy response model (SARM) introduced by Maris and van der Maas (Psychometrika 77:615-633, 2012). In these models, the scores that result from a scoring rule that incorporates both the speed and accuracy of item responses are modeled. Our generalization is similar to that of the one-parameter logistic (or Rasch) model to the two-parameter logistic (or Birnbaum) model in item response theory. An expectation-maximization (EM) algorithm for estimating model parameters and standard errors was developed. Furthermore, methods to assess model fit are provided in the form of generalized residuals for item score functions and saddlepoint approximations to the density of the sum score. The presented methods were evaluated in a small simulation study, the results of which indicated good parameter recovery and reasonable type I error rates for the residuals. Finally, the methods were applied to two real data sets. It was found that the two-parameter SARM showed improved fit compared to the one-parameter SARM in both data sets.

A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing.

We compare three modelling frameworks for accuracy and speed of item responses in the context of adaptive testing. The first framework is based on modelling scores that result from a scoring rule that incorporates both accuracy and speed. The second framework is the hierarchical modelling approach developed by van der Linden (2007, Psychometrika, 72, 287) in which a regular item response model is specified for accuracy and a log-normal model for speed. The third framework is the diffusion framework in which the response is assumed to be the result of a Wiener process. Although the three frameworks differ in the relation between accuracy and speed, one commonality is that the marginal model for accuracy can be simplified to the two-parameter logistic model. We discuss both conditional and marginal estimation of model parameters. Models from all three frameworks were fitted to data from a mathematics and spelling test. Furthermore, we applied a linear and adaptive testing mode to the data off-line in order to determine differences between modelling frameworks. It was found that a model from the scoring rule framework outperformed a hierarchical model in terms of model-based reliability, but the results were mixed with respect to correlations with external measures.

An Evaluation of Different Statistical Targets for Assembling Parallel Forms in Item Response Theory.

Assembly of parallel forms is an important step in the test development process. Therefore, choosing a suitable theoretical framework to generate well-defined test specifications is critical. The performance of different statistical targets of test specifications using the test characteristic curve (TCC) and the test information function (TIF) was investigated. Test length, the number of test forms, and content specifications are considered as well. The TCC target results in forms that are parallel in difficulty, but not necessarily in terms of precision. Vice versa, test forms created using a TIF target are parallel in terms of precision, but not necessarily in terms of difficulty. As sometimes the focus is either on TIF or TCC, differences in either difficulty or precision can arise. Differences in difficulty can be mitigated by equating, but differences in precision cannot. In a series of simulations using a real item bank, the two-parameter logistic model, and mixed integer linear programming for automated test assembly, these differences were found to be quite substantial. When both TIF and TCC are combined into one target with manipulation to relative importance, these differences can be made to disappear.

On the explaining-away phenomenon in multivariate latent variable models.

Many probabilistic models for psychological and educational measurements contain latent variables. Well-known examples are factor analysis, item response theory, and latent class model families. We discuss what is referred to as the 'explaining-away' phenomenon in the context of such latent variable models. This phenomenon can occur when multiple latent variables are related to the same observed variable, and can elicit seemingly counterintuitive conditional dependencies between latent variables given observed variables. We illustrate the implications of explaining away for a number of well-known latent variable models by using both theoretical and real data examples.

Empirical recovery of argumentation learning progressions in scenario-based assessments of English language arts / Un estudio de casos en el diseño de la evaluación centrada en principios: diseño de evaluaciones para medir e impulsar el desarrollo de la argumentación en habilidades de lecto-escritura

We investigate methods for studying learning progressions in English language arts using data from scenario-based assessments. Particularly, our interest lies in the empirical recovery of learning progressions in argumentation for middle school students. We collected data on three parallel assessment forms that consist of scenario-based task sets with multiple item formats, where students randomly took two of the three assessments. We fitted several item response theory models, and used model-based measures to classify students into levels of the argumentation learning progression. Although there were some differences in difficulty between parallel tasks, good agreement was found among the classifications of the parallel forms. Overall, we managed to recover empirically the order of the levels in the argumentation learning progression as they were assigned to tasks of the assessments by the theoretical framework

En este trabajo se investigan métodos para estudiar las progresiones de aprendizaje de competencias de lecto-escritura utilizando evaluaciones basadas en escenarios. En particular, nos interesa poder replicar las progresiones en el aprendizaje de la capacidad para argumentar en estudiantes de enseñanza secundaria obligatoria. Se han recogido datos aplicando tres formas paralelas de una prueba que consiste en conjuntos de tareas basadas en escenarios con preguntas de distinto formato; cada estudiante respondió a dos de estas tres formas, que fueron asignadas aleatoriamente a cada uno de ellos. Se han ajustado a los datos varios modelos de teoría de respuesta al ítem y se han utilizado medidas basadas en esta teoría para clasificar a los estudiantes en los niveles correspondientes de la progresión en el aprendizaje de la capacidad para argumentar. Aunque se han detectado algunas diferencias en las tareas de las formas paralelas, se ha encontrado un grado razonable de acuerdo entre las clasificaciones realizadas en base a las formas paralelas de la prueba. En general, se ha replicado empíricamente el orden de los niveles de la progresión en el aprendizaje de la argumentación, tal y como fueron asignados los niveles a las tareas de la prueba en el marco teórico

The shortened Raven Standard Progressive Matrices: item response theory-based psychometric analyses and normative data.

The purpose of the present study was to evaluate the psychometric properties of a shortened version of the Raven Standard Progressive Matrices (SPM) under an item response theory framework (the one- and two-parameter logistic models). The shortened Raven SPM was administered to N = 453 cognitively healthy adults aged between 24 and 83 years. The IQ point estimates that were obtained under the one- and two-parameter logistic models were very similar (r = .97), but the two-parameter logistic-based test version had a higher measurement precision. The results showed that older age and being female were associated with a lower Raven SPM test performance. A user-friendly computer program was provided to facilitate the scoring and norming of the shortened Raven SPM under the different frameworks.