RESUMO
Recognizing hand-object interactions presents a significant challenge in computer vision. It arises due to the varying nature of hand-object interactions. Moreover, estimating the 3D position of a hand from a single frame can be problematic, especially when the hand obstructs the view of the object from the observer's perspective. In this article, we present a novel approach to recognizing objects and facilitating virtual interactions, using a steering wheel as an illustrative example. We propose a real-time solution for identifying hand-object interactions in eXtended reality (XR) environments. Our approach relies on data captured by a single RGB camera during a manipulation scenario involving a steering wheel. Our model pipeline consists of three key components: (a) a hand landmark detector based on the MediaPipe cross-platform hand tracking solution; (b) a three-spoke steering wheel model tracker implemented using the faster region-based convolutional neural network (Faster R-CNN) architecture; and (c) a gesture recognition module designed to analyze interactions between the hand and the steering wheel. This approach not only offers a realistic experience of interacting with steering-based mechanisms but also contributes to reducing emissions in the real-world environment. Our experimental results demonstrate the natural interaction between physical objects in virtual environments, showcasing precision and stability in our system.
RESUMO
The worldwide COVID-19 pandemic has changed the development plans of every country. Instead, governments had to constantly deal with ever-emerging issues in healthcare, education, economics and industry. As a result, there has been an accelerated introduction of digitalization in these spheres. Thus, an increasing number of people have started using electronic services that have improved their digital literacy. This feature has had a positive impact on society and helped to create new interaction tools between populations and governments, students and institutions, customers and companies. The article aims to analyze how studying radio electronics can be improved by involving new tools and how they can be applied in distance learning. This work presents the results of the development and application of a virtual radio signal simulation in the educational process in the form of a laboratory practicum. Working on this approach required specific research in the field; the foreign experience was observed and studied. The review allowed us to find out how digitalization and the application of digital tools affect the behaviour, cognition, and overall performance of students during the pandemic. The authors conducted a questionnaire among students to evaluate the features of the virtual laboratory work and their effect on the educational process. The results analyzed are given in the article. They showed that students highly appreciate the introduction of such tools in learning. Moreover, like the entire laboratory, the proposed model can be used in the educational process offline and with distance learning. Finally, the article describes the experience and results of the software package's development and integration for the spectrum analyzer's computer model and virtual laboratory work using the MS VS environment in C ++. The results of the conducted work demonstrate the versatility of the proposed approach, its positive impact on the educational process, high potential in the other spheres.
