Artificial Intelligence-Based Smart Tele-Assisting Technology for First-Year Engineering Students

ABSTRACT

Engineering programs emphasize students career advancement by ensuring that engineering students gain technical and professional capabilities during their four-year study. In a traditional engineering laboratory, students "learn by doing", and laboratory equipment facilitates their discipline-specific knowledge acquisition. Unfortunately, there were significant educational uncertainties, such as COVID-19, which halted laboratory activities for an extended period, causing challenges for students to perform and obtain practical experiments on campus. To overcome these challenges, this research proposes and develops an Artificial Intelligence-based smart tele-assisting technology application to digitalize first-year engineering students practical experience by incorporating Augmented Reality (AR) and Machine Learning (ML) algorithms using the HoloLens 2. This application improves virtual procedural demonstrations and assists first-year engineering students in conducting practical activities remotely. This research also applies various machine learning algorithms to identify and classify different images of electronic components and detect the positions of each component on the breadboard (using the HoloLens 2). Based on a comparative analysis of machine learning algorithms, a hybrid CNN-SVM (Convolutional Neural Network - Support Vector Machine) model is developed and is observed that a hybrid model provides the highest average prediction accuracy compared to other machine learning algorithms. With the help of AR (HoloLens 2) and the hybrid CNN-SVM model, this research allows students to reduce component placement errors on a breadboard and increases students competencies, decision-making abilities, and technical skills to conduct simple laboratory practices remotely. © 2023 IEEE.