Examining the validity of chiropractic grade point averages for predicting National Board of Chiropractic Examiners Part I exam scores.

OBJECTIVE: The main objective of this study was to evaluate the validity of grade point average (GPA) for predicting the National Board of Chiropractic Examiners (NBCE) Part I exam scores using chiropractic GPA. METHODS: Data were collected during the January 2019 computer-based testing administration of the NBCE's Part I exam. The sample size was n = 2278 of test takers from 18 domestic and 4 international chiropractic educational institutions. Six regression models were developed and tested to predict the Part I domain scores from chiropractic GPA while controlling for self-reported demographic variables. Residuals from the models were disaggregated by pre-chiropractic GPA. RESULTS: Chiropractic GPA revealed a positive, statistically significant correlation with sex. The chiropractic GPA was found to be a significant predictor of the Part I domain scores. A different perspective was obtained when residuals (observed minus predicted) were collected and split by the pre-chiropractic GPA. Very good students tended to be underpredicted, while other students were overpredicted. CONCLUSION: This study builds on the cascading evidence from educational literature by providing additional results suggesting that undergraduate (prechiropractic) GPA as well as the GPA obtained in doctor of chiropractic programs are related to the future performance on the NBCE Part I exam. The results provide a first glance at the connection between the standardized test scores, which are often used for instructors' and institutional evaluation and the GPA obtained in a doctor of chiropractic program.

Practice analysis and changes to the Chiropractic Board of Clinical Nutrition diplomate exam.

OBJECTIVE: The main objective of this study was to report results of the practice analysis survey and to provide insights into the average levels of performance and the importance of professional tasks executed by chiropractic nutritionists. In addition, this study informs the chiropractic community of the changes made to the Chiropractic Board of Clinical Nutrition diplomate exam. METHODS: Seventy-eight practicing chiropractic nutritionists responded to the practice analysis survey. Their responses were analyzed, and conclusions about frequency and importance of performance tasks were reached. A panel of subject matter experts provided a qualitative review of the survey responses. The quantitative and qualitative analyses of the survey responses indicated that minor changes to the test plan were needed. RESULTS: Descriptive statistical techniques were employed to analyze the survey responses. The qualitative panel suggested reducing the number of domains on the nutrition exam from 7 to 6 by combining Laboratory and Nutrition-Specific Testing and Imaging and Other Special Studies domains. Additionally, the panel decided on the final distribution of weights combining the quantitative results with qualitative perspectives. CONCLUSION: The practice analysis is a first step in the definition of the skills required for practicing chiropractic nutritionists. The analysis becomes one of the references and a decision-making tool used by the board for developing and administrating quality assessments.

A Two-Level Alternating Direction Model for Polytomous Items With Local Dependence.

The chiropractic clinical competency examination uses groups of items that are integrated by a common case vignette. The nature of the vignette items violates the assumption of local independence for items nested within a vignette. This study examines via simulation a new algorithmic approach for addressing the local independence violation problem using a two-level alternating directions testlet model. Parameter values for item difficulty, discrimination, test-taker ability, and test-taker secondary abilities associated with a particular testlet are generated and parameter recovery through Markov Chain Monte Carlo Bayesian methods and generalized maximum likelihood estimation methods are compared. To aid with the complex computational efforts, the novel so-called TensorFlow platform is used. Both estimation methods provided satisfactory parameter recovery, although the Bayesian methods were found to be somewhat superior in recovering item discrimination parameters. The practical significance of the results are discussed in relation to obtaining accurate estimates of item, test, ability parameters, and measurement reliability information.

The transition to digital presentation of the diagnostic imaging domain of the Part IV examination of the National Board of Chiropractic Examiners.

OBJECTIVE: This article introduces changes made to the diagnostic imaging (DIM) domain of the Part IV of the National Board of Chiropractic Examiners examination and evaluates the effects of these changes in terms of item functioning and examinee performance. METHODS: To evaluate item function, classical test theory and item response theory (IRT) methods were employed. Classical statistics were used for the assessment of item difficulty and the relation to the total test score. Item difficulties along with item discrimination were calculated using IRT. We also studied the decision accuracy of the redesigned DIM domain. RESULTS: The diagnostic item analysis revealed similarity in item function across test forms and across administrations. The IRT models found a reasonable fit to the data. The averages of the IRT parameters were similar across test forms and across administrations. The classification of test takers into ability (theta) categories was consistent across groups (both norming and all examinees), across all test forms, and across administrations. CONCLUSION: This research signifies a first step in the evaluation of the transition to digital DIM high-stakes assessments. We hope that this study will spur further research into evaluations of the ability to interpret radiographic images. In addition, we hope that the results prove to be useful for chiropractic faculty, chiropractic students, and the users of Part IV scores.

Current state and future directions of the National Board of Chiropractic Examiners.

The objective of this paper is to describe changes made to chiropractic national board examinations in the United States, including methodologies in test scoring, and to discuss future directions in test development and administration being considered by the National Board of Chiropractic Examiners (NBCE). Additionally, this paper serves as an introduction to the articles written by the NBCE staff and published in this issue of the journal. Statistical perspective on the properties of a test are presented, and reasons for the NBCE moving to item response theory for test scoring are described. NBCE consideration of on-demand testing and changes implemented in the Part IV practical examination are also discussed.

Score production and quantitative methods used by the National Board of Chiropractic Examiners for postexam analyses.

OBJECTIVE: The National Board of Chiropractic Examiners (NBCE) uses a robust system for data analysis. The aim of this work is to introduce the reader to the process of score production and the quantitative methods used by the psychometrician and data analysts of the NBCE. METHODS: The NBCE employs data validation, diagnostic analyses, and item response theory-based modeling of responses to estimate test takers' abilities and item-related parameters. For this article, the authors generated 1303 synthetic item responses to 20 multiple-choice items with 4 response options to each item. These data were used to illustrate and explain the processes of data validation, diagnostic item analysis, and item calibration based on item response theory. RESULTS: The diagnostic item analysis is presented for items 1 and 5 of the data set. The 3-parameter logistic item response theory model was used for calibration. Numerical and graphical results are presented and discussed. CONCLUSION: Demands for data-driven decision making and evidence-based effectiveness create a need for objective measures to be used in educational program reviews and evaluations. Standardized test scores are often included in that array of objective measures. With this article, we offer transparency of score production used for NBCE testing.

Variables associated with successful performance on the National Board of Chiropractic Examiners Part IV examination.

OBJECTIVE: The objectives of this study were to (1) identify factors predictive of performance on the National Board of Chiropractic Examiners Part IV exam and (2) investigate correlations between the scores obtained in the Part I, Part II, Physiotherapy, and Part III exams and the Part IV examination. METHODS: A random sample of 1341 records was drawn from National Board of Chiropractic Examiners data to investigate the relationships between the scores obtained on the National Board of Chiropractic Examiners exams. A hierarchical multiple regression analysis related the performance on Part IV to examinee's gender, Part IV repeater status, and scores obtained on the Part I, Part II, Physiotherapy, and Part III exams. RESULTS: The analyses revealed statistical relations among all National Board of Chiropractic Examiners exams. The correlations between Part IV and Part I ranged from r = .31 to r = .4; between Part IV and Part II from r = .34 to r = .45. The correlation between Part IV and Physiotherapy was r = .44; between Part IV and Part III was r = .46. The strongest predictors of the Part IV score were found to be examinees' scores in Diagnostic Imaging, ß̂ = .19, p < .001; Chiropractic Practice, ß̂ = .17, p < .001; Physiotherapy, ß̂ = .15, p < .001; and the Part III exam ß̂ = .19, p < .001. CONCLUSIONS: Performance on the National Board of Chiropractic Examiners Part IV examination is related to the performance in all other National Board of Chiropractic Examiners exams.

A primer on standardized testing: History, measurement, classical test theory, item response theory, and equating.

OBJECTIVE: This article presents health science educators and researchers with an overview of standardized testing in educational measurement. The history, theoretical frameworks of classical test theory, item response theory (IRT), and the most common IRT models used in modern testing are presented. METHODS: A narrative overview of the history, theoretical concepts, test theory, and IRT is provided to familiarize the reader with these concepts of modern testing. Examples of data analyses using different models are shown using 2 simulated data sets. One set consisted of a sample of 2000 item responses to 40 multiple-choice, dichotomously scored items. This set was used to fit 1-parameter logistic (PL) model, 2PL, and 3PL IRT models. Another data set was a sample of 1500 item responses to 10 polytomously scored items. The second data set was used to fit a graded response model. RESULTS: Model-based item parameter estimates for 1PL, 2PL, 3PL, and graded response are presented, evaluated, and explained. CONCLUSION: This study provides health science educators and education researchers with an introduction to educational measurement. The history of standardized testing, the frameworks of classical test theory and IRT, and the logic of scaling and equating are presented. This introductory article will aid readers in understanding these concepts.

Theory of mind and executive function in Chinese preschool children.

Cross-cultural research on children's theory of mind (ToM) understanding has raised questions about its developmental sequence and relationship with executive function (EF). The current study examined how ToM develops (using the tasks from Wellman & Liu, 2004) in relation to 2 EF skills (conflict inhibition, working memory) in 997 Chinese preschoolers (ages 3, 4, 5) in Chengdu, China. Compared with prior research with other Chinese and non-Chinese children, some general patterns in development were replicated in this sample. However, the children showed culture-specific reversals in the developmental sequence of ToM. For example, Chengdu children performed differently on the 2 false-belief tasks that were thought to be equivalent. Furthermore, conflict inhibition as well as working memory uniquely predicted ToM performance. We discuss the issues of ToM development as they relate to test items and cross-cultural--and subcultural--differences.