Using latent variable models to make gaming-the-system detection robust to context variations.

Gaming the system, a behavior in which learners exploit a system's properties to make progress while avoiding learning, has frequently been shown to be associated with lower learning. However, when we applied a previously validated gaming detector across conditions in experiments with an algebra tutor, the detected gaming was not associated with reduced learning, challenging its validity in our study context. Our exploratory data analysis suggested that varying contextual factors across and within conditions contributed to this lack of association. We present a new approach, latent variable-based gaming detection (LV-GD), that controls for contextual factors and more robustly estimates student-level latent gaming tendencies. In LV-GD, a student is estimated as having a high gaming tendency if the student is detected to game more than the expected level of the population given the context. LV-GD applies a statistical model on top of an existing action-level gaming detector developed based on a typical human labeling process, without additional labeling effort. Across three datasets, we find that LV-GD consistently outperformed the original detector in validity measured by association between gaming and learning as well as reliability. LV-GD also afforded high practical utility: it more accurately revealed intervention effects on gaming, revealed a correlation between gaming and perceived competence in math and helped understand productive detected gaming behaviors. Our approach is not only useful for others wanting a cost-effective way to adapt a gaming detector to their context but is also generally applicable in creating robust behavioral measures.

Worked examples and tutored problem solving: redundant or synergistic forms of support?

The current research investigates a combination of two instructional approaches, tutored problem solving and worked examples. Tutored problem solving with automated tutors has proven to be an effective instructional method. Worked-out examples have been shown to be an effective complement to untutored problem solving, but it is largely unknown whether they are an effective complement to tutored problem solving. Further, while computer-based learning environments offer the possibility of adaptively transitioning from examples to problems while tailoring to an individual learner, the effectiveness of such machine-adapted example fading is largely unstudied. To address these research questions, one lab and one classroom experiment were conducted. Both studies compared a standard Cognitive Tutor with two example-enhanced Cognitive Tutors, in which the fading of worked-out examples occurred either in a fixed way or adaptively. Results indicate that the adaptive fading of worked-out examples leads to higher transfer performance on delayed posttests than the other two methods.