Causal Analysis of On-line Math Tutoring Impact on Low-income High School Students Using Bayesian Logistic and Beta Regressions

ABSTRACT

The use of on-line tutoring, especially after the COVID-19 pandemic, has increased dramatically. It has become clear that measuring the effectiveness of on-line tutoring, especially on low-income students, is much needed in such difficult times. This paper, which is based on observational data collected before the COVID-19 era, is targeting measuring the impact of a web-based math tutoring program, Noon Academy, on the academic achievement of low-income high school students (grades 10 to 12) in Saudi Arabia. We use a large amount of data collected in a student registration process and two Bayesian generalized linear models (GLM) to measure the tutoring causal effects. Model 1 uses a binomial logistic regression to predict the impact of enrolling in the tutoring program on the rate of passing in a number of students. Model 2 uses a multi-level Beta regression to measure the impact of the number of minutes on the total mark. Model 1 results show that giving math tutoring to higher-failing-risk students significantly improves the rate of passing by +5%, reaching a maximum of +17.15% in some classes of students. Model 2 shows a significant positive impact of the number of tutoring minutes on the yearly math mark (max of 100), reaching an average of +3.52 marks for the highest number of minutes taken. The paper presents an application of a causal analysis approaches on a real-life social problem. It demonstrates how the model is used to obtain a measure of the impact with quantifiable uncertainty that can be used in practice.