Multidimensional Item Response Theory in the Style of Collaborative Filtering.

This paper presents a machine learning approach to multidimensional item response theory (MIRT), a class of latent factor models that can be used to model and predict student performance from observed assessment data. Inspired by collaborative filtering, we define a general class of models that includes many MIRT models. We discuss the use of penalized joint maximum likelihood to estimate individual models and cross-validation to select the best performing model. This model evaluation process can be optimized using batching techniques, such that even sparse large-scale data can be analyzed efficiently. We illustrate our approach with simulated and real data, including an example from a massive open online course. The high-dimensional model fit to this large and sparse dataset does not lend itself well to traditional methods of factor interpretation. By analogy to recommender-system applications, we propose an alternative "validation" of the factor model, using auxiliary information about the popularity of items consulted during an open-book examination in the course.

Summative assessments of web-based patient simulations of pre-clinical local anaesthesia and non-surgical extraction.

INTRODUCTION: The purpose of this study was to evaluate the use of web-based patient simulations, a whole-task simulation, in oral surgery education for second-year pre-clinical dental students on learning outcomes regarding cognitive, psychomotor and professional interpersonal skills for two topics: local anaesthesia and non-surgical extractions. MATERIALS AND METHODS: In 2018-19, using an experimental design in a live course, we evaluated two web-based patient simulations on student learning outcomes in an introductory course on oral maxillofacial surgery. The study was repeated across two semesters, Fall (N = 109) and Spring (N = 112), on two different topics, namely local anaesthesia and extractions, respectively. Learning outcomes were evaluated for each topic with two different assessment formats: a student-recorded role-play video project and a fifty-item multiple-choice test. RESULTS: For both topics, local anaesthesia and non-surgical extractions, students in the group, Web Sim, who used the web-based patient simulation over and above the online lesson material earned significantly higher scores in the role-play patient video project compared with the group, No Web Sim, who only used the online lesson materials. However, scores on the fifty-item multiple-choice test did not differ significantly between groups. CONCLUSIONS: The results suggest that the web-based patient simulation as a formative assessment type is a useful scaffolding tool for the summative student-recorded role-play patient video project due to the similarity in activity types that support clinical learning objectives.

Psychometric Models of Small Group Collaborations.

The social combination theory of group problem solving is used to extend existing psychometric models to collaborative settings. A model for pairwise group work is proposed, the implications of the model for assessment design are considered, and its estimation is addressed. The results are illustrated with an empirical example in which dyads work together on a twelfth-grade level mathematics assessment. In conclusion, attention is given to avenues of research that seem most fruitful for advancing current initiatives concerning the assessment of collaboration, teamwork, and related constructs.