Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques.

Virtualization plays a critical role in enriching the user experience in Virtual Reality (VR) by offering heightened realism, increased immersion, safer navigation, and newly achievable levels of interaction and personalization, specifically in indoor environments. Traditionally, the creation of virtual content has fallen under one of two broad categories: manual methods crafted by graphic designers, which are labor-intensive and sometimes lack precision; traditional Computer Vision (CV) and Deep Learning (DL) frameworks that frequently result in semi-automatic and complex solutions, lacking a unified framework for both 3D reconstruction and scene understanding, often missing a fully interactive representation of the objects and neglecting their appearance. To address these diverse challenges and limitations, we introduce the Virtual Experience Toolkit (VET), an automated and user-friendly framework that utilizes DL and advanced CV techniques to efficiently and accurately virtualize real-world indoor scenarios. The key features of VET are the use of ScanNotate, a retrieval and alignment tool that enhances the precision and efficiency of its precursor, supported by upgrades such as a preprocessing step to make it fully automatic and a preselection of a reduced list of CAD to speed up the process, and the implementation in a user-friendly and fully automatic Unity3D application that guides the users through the whole pipeline and concludes in a fully interactive and customizable 3D scene. The efficacy of VET is demonstrated using a diversified dataset of virtualized 3D indoor scenarios, supplementing the ScanNet dataset.

Hybrid brain/neural interface and autonomous vision-guided whole-arm exoskeleton control to perform activities of daily living (ADLs).

BACKGROUND: The aging of the population and the progressive increase of life expectancy in developed countries is leading to a high incidence of age-related cerebrovascular diseases, which affect people's motor and cognitive capabilities and might result in the loss of arm and hand functions. Such conditions have a detrimental impact on people's quality of life. Assistive robots have been developed to help people with motor or cognitive disabilities to perform activities of daily living (ADLs) independently. Most of the robotic systems for assisting on ADLs proposed in the state of the art are mainly external manipulators and exoskeletal devices. The main objective of this study is to compare the performance of an hybrid EEG/EOG interface to perform ADLs when the user is controlling an exoskeleton rather than using an external manipulator. METHODS: Ten impaired participants (5 males and 5 females, mean age 52 ± 16 years) were instructed to use both systems to perform a drinking task and a pouring task comprising multiple subtasks. For each device, two modes of operation were studied: synchronous mode (the user received a visual cue indicating the sub-tasks to be performed at each time) and asynchronous mode (the user started and finished each of the sub-tasks independently). Fluent control was assumed when the time for successful initializations ranged below 3 s and a reliable control in case it remained below 5 s. NASA-TLX questionnaire was used to evaluate the task workload. For the trials involving the use of the exoskeleton, a custom Likert-Scale questionnaire was used to evaluate the user's experience in terms of perceived comfort, safety, and reliability. RESULTS: All participants were able to control both systems fluently and reliably. However, results suggest better performances of the exoskeleton over the external manipulator (75% successful initializations remain below 3 s in case of the exoskeleton and bellow 5s in case of the external manipulator). CONCLUSIONS: Although the results of our study in terms of fluency and reliability of EEG control suggest better performances of the exoskeleton over the external manipulator, such results cannot be considered conclusive, due to the heterogeneity of the population under test and the relatively limited number of participants.

Towards the Automation of Infrared Thermography Inspections for Industrial Maintenance Applications.

The maintenance of industrial equipment extends its useful life, improves its efficiency, reduces the number of failures, and increases the safety of its use. This study proposes a methodology to develop a predictive maintenance tool based on infrared thermographic measures capable of anticipating failures in industrial equipment. The thermal response of selected equipment in normal operation and in controlled induced anomalous operation was analyzed. The characterization of these situations enabled the development of a machine learning system capable of predicting malfunctions. Different options within the available conventional machine learning techniques were analyzed, assessed, and finally selected for electronic equipment maintenance activities. This study provides advances towards the robust application of machine learning combined with infrared thermography and augmented reality for maintenance applications of industrial equipment. The predictive maintenance system finally selected enables automatic quick hand-held thermal inspections using 3D object detection and a pose estimation algorithm, making predictions with an accuracy of 94% at an inference time of 0.006 s.

Small flexible automated system for monitoring Caenorhabditis elegans lifespan based on active vision and image processing techniques.

Traditionally Caenorhabditis elegans lifespan assays are performed by manually inspecting nematodes with a dissection microscope, which involves daily counting of live/dead worms cultured in Petri plates for 21-25 days. This manual inspection requires the screening of hundreds of worms to ensure statistical robustness, and is therefore a time-consuming approach. In recent years, various automated artificial vision systems have been reported to increase the throughput, however they usually provide less accurate results than manual assays. The main problems identified when using these vision systems are the false positives and false negatives, which occur due to culture media changes, occluded zones, dirtiness or condensation of the Petri plates. In this work, we developed and described a new C. elegans monitoring machine, SiViS, which consists of a flexible and compact platform design to analyse C. elegans cultures using the standard Petri plates seeded with E. coli. Our system uses an active vision illumination technique and different image-processing pipelines for motion detection, both previously reported, providing a fully automated image processing pipeline. In addition, this study validated both these methods and the feasibility of the SiViS machine for lifespan experiments by comparing them with manual lifespan assays. Results demonstrated that the automated system yields consistent replicates (p-value log rank test 0.699), and there are no significant differences between automated system assays and traditionally manual assays (p-value 0.637). Finally, although we have focused on the use of SiViS in longevity assays, the system configuration is flexible and can, thus, be adapted to other C. elegans studies such as toxicity, mobility and behaviour.

Improving lifespan automation for Caenorhabditis elegans by using image processing and a post-processing adaptive data filter.

Automated lifespan determination for C. elegans cultured in standard Petri dishes is challenging. Problems include occlusions of Petri dish edges, aggregation of worms, and accumulation of dirt (dust spots on lids) during assays, etc. This work presents a protocol for a lifespan assay, with two image-processing pipelines applied to different plate zones, and a new data post-processing method to solve the aforementioned problems. Specifically, certain steps in the culture protocol were taken to alleviate aggregation, occlusions, contamination, and condensation problems. This method is based on an active illumination system and facilitates automated image sequence analysis, does not need human threshold adjustments, and simplifies the techniques required to extract lifespan curves. In addition, two image-processing pipelines, applied to different plate zones, were employed for automated lifespan determination. The first image-processing pipeline was applied to a wall zone and used only pixel level information because worm size or shape features were unavailable in this zone. However, the second image-processing pipeline, applied to the plate centre, fused information at worm and pixel levels. Simple death event detection was used to automatically obtain lifespan curves from the image sequences that were captured once daily throughout the assay. Finally, a new post-processing method was applied to the extracted lifespan curves to filter errors. The experimental results showed that the errors in automated counting of live worms followed the Gaussian distribution with a mean of 2.91% and a standard deviation of ±12.73% per Petri plate. Post-processing reduced this error to 0.54 ± 8.18% per plate. The automated survival curve incurred an error of 4.62 ± 2.01%, while the post-process method reduced the lifespan curve error to approximately 2.24 ± 0.55%.

Intelligent Multimodal Framework for Human Assistive Robotics Based on Computer Vision Algorithms.

Assistive technologies help all persons with disabilities to improve their accessibility in all aspects of their life. The AIDE European project contributes to the improvement of current assistive technologies by developing and testing a modular and adaptive multimodal interface customizable to the individual needs of people with disabilities. This paper describes the computer vision algorithms part of the multimodal interface developed inside the AIDE European project. The main contribution of this computer vision part is the integration with the robotic system and with the other sensory systems (electrooculography (EOG) and electroencephalography (EEG)). The technical achievements solved herein are the algorithm for the selection of objects using the gaze, and especially the state-of-the-art algorithm for the efficient detection and pose estimation of textureless objects. These algorithms were tested in real conditions, and were thoroughly evaluated both qualitatively and quantitatively. The experimental results of the object selection algorithm were excellent (object selection over 90%) in less than 12 s. The detection and pose estimation algorithms evaluated using the LINEMOD database were similar to the state-of-the-art method, and were the most computationally efficient.

Predicting Gilthead Sea Bream (Sparus aurata) Freshness by a Novel Combined Technique of 3D Imaging and SW-NIR Spectral Analysis.

A technique that combines the spatial resolution of a 3D structured-light (SL) imaging system with the spectral analysis of a hyperspectral short-wave near infrared system was developed for freshness predictions of gilthead sea bream on the first storage days (Days 0-6). This novel approach allows the hyperspectral analysis of very specific fish areas, which provides more information for freshness estimations. The SL system obtains a 3D reconstruction of fish, and an automatic method locates gilthead's pupils and irises. Once these regions are positioned, the hyperspectral camera acquires spectral information and a multivariate statistical study is done. The best region is the pupil with an R² of 0.92 and an RMSE of 0.651 for predictions. We conclude that the combination of 3D technology with the hyperspectral analysis offers plenty of potential and is a very promising technique to non destructively predict gilthead freshness.