Learning Management System Analytics to Examine the Behavior of Students in High Enrollment STEM Courses During the Transition to Online Instruction

ABSTRACT

The emergence of the COVID-19 pandemic resulted in the transition to near-total online instruction in early 2020. Several studies surveyed students about the impact of the pandemic on their behavior and engagement with their education;however, those studies may not include analysis of actual student behaviors. The field of learning analytics allows researchers to examine the records made while students interact with the educational technology tools that are commonly used to facilitate instruction in Institutes of Higher Education (IHEs). In most universities, delivery of many instructional materials is conducted via a Learning Management System (LMS). In this study, we describe our process to examine student interactions within the LMS to discover any measurable changes to student behavior during the pandemic.We examined the usage logs of the Canvas LMS at a large university in the midwestern US to examine the behavior of students' interactions with high-enrollment STEM courses in two semesters one prior to and one during the pandemic. The log data was integrated with student demographic data so that the LMS behavior of subsets of students can be compared. Machine learning algorithms including clustering models and association rule mining were applied on the data. The results of this study demonstrate that the students' behaviors did change in the transition to online instruction. Students had more frequent sessions in the LMS on both computers and mobile devices, although the duration of their mobile sessions was shorter after courses were moved online. Further, students in historically underrepresented groups in STEM fields were found to use their mobile devices more frequently for academic work. The information uncovered in this study can be used to inform future instructional design practices with the LMS to promote an equitable experience for all students. © 2022 IEEE.