Assessing the Causal Impact of Online Instruction due to COVID-19 on Students' Grades and its aftermath on Grade Prediction Models
2nd ACM Conference on Information Technology for Social Good, GoodIT 2022
; : 32-38, 2022.
Article
in English
| Scopus | ID: covidwho-2053341
ABSTRACT
The COVID-19 pandemic forced many educational institutions to transition to online learning activities. This significantly impacted various aspects of students' lives. Many of the studies aimed at assessing the impact of the online instruction on students' wellbeing and performance have mainly focused on issues such as mental health. However, the impact on student grades-a key measure of student success-has been given little attention. The handful existing studies are either focused on primary schools-where the dynamics are different from higher education-or based on statistical correlations, which are usually not causally rigorous, therefore, prone to biased estimates due to various confounding variables. There are many variables associated with students' grades, thus, to assess the causal impact of the online instruction on students' grades, there is a need for a causally-grounded approach that can control for confounding variables. To that end, we use a causal tree to investigate the impact of online instruction on the grades of the general population as well as different demographic subgroups. Our analysis is based on the demographic and engagement data for the 2019 (offline/control) and 2020 (online/treatment) cohorts of 3 mandatory courses in an Australian university. For all 3 courses, our results show that for any given student in the population, the average grade they would have gotten, had they studied offline, reduced by 3.6%, 4.7%, and 14% respectively. Further analyses show that among students with similar level of (low) engagement with the virtual learning environment, the average grade international students would have gotten, had they studied face-to-face, reduced by 19.9%, 36.6%, and 46.9% more than their domestic counterparts despite having similar engagement for the 3 courses respectively. These subgroup disparities have the potential to exacerbate existing inequalities. Given the current concerns about algorithmic bias in learning analytics (LA), we trained grade prediction models with the data and investigated for algorithmic bias. Interestingly, we find that by simply changing citizenship status, a student gets a new predicted grade, entirely different from what was initially predicted given their actual citizenship status. This implies that researchers must be careful when building LA models on COVID-19 era data. © 2022 ACM.
Algorithmic bias; Causal analysis; COVID-19; Inequality; Learning analytics; Virtual learning environment; Computer aided instruction; E-learning; Education computing; Population statistics; Students; Algorithmics; Grade predictions; Learning analytic; Online instructions; Prediction modelling; Students' grades; Virtual learning environments
Full text:
Available
Collection:
Databases of international organizations
Database:
Scopus
Type of study:
Experimental Studies
/
Prognostic study
Language:
English
Journal:
2nd ACM Conference on Information Technology for Social Good, GoodIT 2022
Year:
2022
Document Type:
Article
Similar
MEDLINE
...
LILACS
LIS