Development and Validation of a Vocabulary Measure in the Mobile Toolbox.

OBJECTIVE: We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies. METHOD: Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones. RESULTS: The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001). CONCLUSION: The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Validade baseada nos processos de resposta do trilhar - instrumento de triagem do vocabulário infantil / Validity evidence based on response processes of the TRILHAR - screening tool for infant vocabulary

RESUMO Objetivo Verificar as evidências de validade baseadas nos processos de resposta de um instrumento de triagem do vocabulário. Método Trata-se de um estudo descritivo, transversal e quantitativo, aplicado em uma amostra de 133 crianças entre 3 e 7 anos de idade, subdivididos em cinco grupos de acordo a faixa etária. O instrumento sobre o qual se trata a pesquisa é o TRILHAR, que visa a triagem do vocabulário receptivo e expressivo, composto por dez fichas de atividades para cada idade-alvo. Os dados coletados focaram no tempo de aplicação em minutos e segundos, compreensão das instruções do teste e comportamento do escolar durante a aplicação. Os dados foram analisados descritivamente em relação a média, desvio-padrão e porcentagem por meio do software IBM SPSS Statistics. Resultados O tempo de aplicação foi curto e diminuiu de acordo com a progressão da idade. O tempo médio de aplicação foi de 05m19s para o G1, 04m29s para o G2, 04m11s para o G3, 03m40s para o G4 e 02m13s para o G5. Apenas um sujeito (4,8%) do G2 necessitou de repetição das instruções no vocabulário receptivo e dois (6,3%) do G4 no vocabulário expressivo. Foi verificada pequena ocorrência de comportamentos como desinteresse, desatenção e agitação. Conclusão Foi verificado que o tempo de aplicação do instrumento necessita de um curto período de tempo, sendo mais dispendiosa no grupo de crianças com três anos de idade. A baixa necessidade de repetição das instruções indica que o instrumento apresenta fácil compreensão por parte da criança.

ABSTRACT Purpose To verify the validity evidence based on response processes of a vocabulary-screening tool. Methods This is a descriptive, cross-sectional and quantitative study, applied in a sample of 133 children between 3 and 7 years of age, divided into five groups, according to their age range. This research evaluates the instrument TRILHAR, which is a screening of receptive and expressive vocabulary, composed by ten activities for each age range. The collected data were the application time in minutes and seconds, screening instructions comprehension and behavior during the screening. The data were analyzed descriptively as mean, standard deviation and percentage using the IBM SPSS Statistics. Results The application time was short, with less time for the greater ages. The mean time of application was 05m19s for G1, 04m29s for G2, 04m11s for G3, 03m40s for G4 and 02m13s for G5. Only one child (4.8%) from G2 required repetition of the instructions for the receptive vocabulary, and two children (6.3%) from G4 for the expressive vocabulary. We observed little occurrence of behaviors like disinterest, inattention and agitation. Conclusion The application of the instrument required a short period, with a longer time for the group of 3-years children. The little necessity of repetition of the instructions indicates that children can easily comprehend the instrument.

Vocabulary Knowledge Predicts Lexical Processing: Evidence from a Group of Participants with Diverse Educational Backgrounds.

Vocabulary knowledge is central to a speaker's command of their language. In previous research, greater vocabulary knowledge has been associated with advantages in language processing. In this study, we examined the relationship between individual differences in vocabulary and language processing performance more closely by (i) using a battery of vocabulary tests instead of just one test, and (ii) testing not only university students (Experiment 1) but young adults from a broader range of educational backgrounds (Experiment 2). Five vocabulary tests were developed, including multiple-choice and open antonym and synonym tests and a definition test, and administered together with two established measures of vocabulary. Language processing performance was measured using a lexical decision task. In Experiment 1, vocabulary and word frequency were found to predict word recognition speed while we did not observe an interaction between the effects. In Experiment 2, word recognition performance was predicted by word frequency and the interaction between word frequency and vocabulary, with high-vocabulary individuals showing smaller frequency effects. While overall the individual vocabulary tests were correlated and showed similar relationships with language processing as compared to a composite measure of all tests, they appeared to share less variance in Experiment 2 than in Experiment 1. Implications of our findings concerning the assessment of vocabulary size in individual differences studies and the investigation of individuals from more varied backgrounds are discussed.

Modeling Floor Effects in Standardized Vocabulary Test Scores in a Sample of Low SES Hispanic Preschool Children under the Multilevel Structural Equation Modeling Framework.

Researchers and practitioners often use standardized vocabulary tests such as the Peabody Picture Vocabulary Test-4 (PPVT-4; Dunn and Dunn, 2007) and its companion, the Expressive Vocabulary Test-2 (EVT-2; Williams, 2007), to assess English vocabulary skills as an indicator of children's school readiness. Despite their psychometric excellence in the norm sample, issues arise when standardized vocabulary tests are used to asses children from culturally, linguistically and ethnically diverse backgrounds (e.g., Spanish-speaking English language learners) or delayed in some manner. One of the biggest challenges is establishing the appropriateness of these measures with non-English or non-standard English speaking children as often they score one to two standard deviations below expected levels (e.g., Lonigan et al., 2013). This study re-examines the issues in analyzing the PPVT-4 and EVT-2 scores in a sample of 4-to-5-year-old low SES Hispanic preschool children who were part of a larger randomized clinical trial on the effects of a supplemental English shared-reading vocabulary curriculum (Pollard-Durodola et al., 2016). It was found that data exhibited strong floor effects and the presence of floor effects made it difficult to differentiate the invention group and the control group on their vocabulary growth in the intervention. A simulation study is then presented under the multilevel structural equation modeling (MSEM) framework and results revealed that in regular multilevel data analysis, ignoring floor effects in the outcome variables led to biased results in parameter estimates, standard error estimates, and significance tests. Our findings suggest caution in analyzing and interpreting scores of ethnically and culturally diverse children on standardized vocabulary tests (e.g., floor effects). It is recommended appropriate analytical methods that take into account floor effects in outcome variables should be considered.