Effects of Eye Vergence and Accommodation on Interactions with Content on an AR Magic-lens Display and its Surroundings.

Augmented reality (AR) magic-lens (ML) displays, such as handheld devices, offer a convenient and accessible way to enrich our environment using virtual imagery. Several display technologies, including conventional monocular, less common stereoscopic, and varifocal displays, are currently being used. Vergence and accommodation effects on depth perception, as well as vergence-accommodation conflict, have been studied, where users interact only with the content on the display. However, little research exists on how vergence and accommodation influence user performance and cognitive-task load when users interact with the content on a display and its surroundings in a short timeframe. Examples of this are validating augmented instructions before making an incision andperforming general hand-eye coordinated tasks such as grasping augmented objects. To improve interactions with future AR displays in such scenarios, we must improve our understanding of this influence. To this end, we conducted two fundamental visual-acuity user studies with 28 and 27 participants, while investigating eye vergence and accommodation distances on four ML displays. Our findings show that minimizing the accommodation difference between the display and its surroundings is crucial when the gaze between the display and its surroundings shifts rapidly. Minimizing the difference in vergence is more important when viewing the display and its surroundings as a single context without shifting the gaze. Interestingly, the vergence-accommodation conflict did not significantly affect the cognitive-task load nor play a pivotal role in the accuracy of interactions with AR ML content and its physical surroundings.

Automatic evaluation-feedback system for automated social skills training.

Social skills training (SST), which is a rehabilitation program for improving daily interpersonal communication, has been used for more than 40 years. Although such training's demand is increasing, its accessibility is limited due to the lack of experienced trainers. To tackle this issue, automated SST systems have been studied for years. An evaluation-feedback pipeline of social skills is a crucial component of an SST system. Unfortunately, research that considers both the evaluation and feedback parts of automation remains insufficient. In this paper, we collected and analyzed the characteristics of a human-human SST dataset that consisted of 19 healthy controls, 15 schizophreniacs, 16 autism spectrum disorder (ASD) participants, and 276 sessions with score labels of six clinical measures. From our analysis of this dataset, we developed an automated SST evaluation-feedback system under the supervision of professional, experienced SST trainers. We identified their preferred or most acceptable feedback methods by running a user-study on the following conditions: with/without recorded video of the role-plays of users and different amounts of positive and corrective feedback. We confirmed a reasonable performance of our social-skill-score estimation models as our system's evaluation part with a maximum Spearman's correlation coefficient of 0.68. For the feedback part, our user-study concluded that people understood more about what aspects they need to improve by watching recorded videos of their own performance. In terms of the amount of feedback, participants most preferred a 2-positive/1-corrective format. Since the average amount of feedback preferred by the participants nearly equaled that from experienced trainers in human-human SSTs, our result suggests the practical future possibilities of an automated evaluation-feedback system that complements SSTs done by professional trainers.

Systematic Review of Augmented Reality Training Systems.

Recent augmented reality (AR) advancements have enabled the development of effective training systems, especially in the medical, rehabilitation, and industrial fields. However, it is unclear from the literature what the intrinsic value of AR to training is and how it differs across multiple application fields. In this work, we gathered and reviewed the prototypes and applications geared towards training the intended user's knowledge, skills, and abilities. Specifically, from IEEE Xplore plus other digital libraries, we collected 64 research papers present in high-impact publications about augmented reality training systems (ARTS). All 64 papers were then categorized according to the training method used, and each paper's evaluations were identified by validity. The summary of the results shows trends in the training methods and evaluations that incorporate ARTS in each field. The narrative synthesis illustrates the different implementations of AR for each of the training methods. In addition, examples of the different evaluation types of the current ARTS are described for each of the aforementioned training methods. We also investigated the different training strategies used by the prevailing ARTS. The insights gleaned from this review can suggest standards for designing ARTS regarding training strategy, and recommendations are provided for the implementation and evaluation of future ARTS.

Cross-talk elimination for lenslet array near eye display based on eye-gaze tracking.

Lenslet array (LA) near-eye displays (NEDs) are a recent technical development that creates a virtual image in the field of view of one or both eyes. A problem occurs when the user's pupil moves out of the LA-NED eye box (i.e., cross-talk) making the image look doubled or ghosted. It negatively impacts the user experience. Although eye-gaze tracking can mitigate this problem, the effect of the solution has not been studied to understand the impact of pupil size and human perception. In this paper, we redefine the cross-talk region as the practical pupil movable region (PPMR50), which differs from eye box size because it considers pupil size and human visual perception. To evaluate the effect of eye-gaze tracking on subjective image quality, three user studies were conducted. From the results, PPMR50 was found to be consistent with human perception, and cross-talk elimination via eye-gaze tracking was better understood in a static gaze scenario. Although the system latency prevented the complete elimination of cross-talk for fast movements or large pupil changes, the problem was greatly alleviated. We also analyzed system delays based on PPMR50, which we newly defined in this paper and provided an optimization scheme to meet the maximum eyeball rotation speed.

Analysis of Feedback Contents and Estimation of Subjective Scores in Social Skills Training.

This paper introduces our analysis results on the feedback contents of Social Skills Training and the consequences of automated score estimation of users' social skills with computational multimodal features. Although previous work showed the possibility of a computerized SST system as a clinical tool, its feedback strategies have not been well-investigated. We focused on the feedback content given by experienced SST trainers in human-human SST sessions to overcome this limitation. We analyzed the points mentioned by experienced SST trainers to determine where they focused during social skills evaluation. We calculated multimodal computational features from video and audio recordings inspired by the results and trained machine learning models for social skills evaluation using these features as input. We trained social skill score prediction models with the highest scores of 0.53 for correlation coefficient and 0.26 for R2. Clinical relevance- We described our automated social skills evaluation method with machine learning models toward a computerized SST system, which can be an additional option to boost the effect of SST by experienced trainers in the future.

Robot touch with speech boosts positive emotions.

A gentle touch is an essential part of human interaction that produces a positive care effect. Previously, robotics studies have shown that robots can reproduce a gentle touch that elicits similar, positive emotional responses in humans. However, whether the positive emotional effects of a robot's touch combined with speech can be enhanced using a multimodal approach remains unclear. This study supports the hypothesis that a multimodal interaction combining gentle touch and speech by a robot enhances positive emotional responses. Here, we conducted an experiment using a robotic arm to perform a gentle touch combined with speech and compared three conditions: touch alone, speech alone, and touch with speech. We assessed participants' subjective ratings of valence, arousal, and human likeliness using subjective emotional responses. Furthermore, we recorded facial electromyography (EMG) from the corrugator supercilii and zygomaticus major muscles and measured skin conductance levels (SCLs) as physiological emotional responses. Our results show that touch combined with speech elicited higher subjective valence and arousal ratings, stronger zygomaticus major EMG and SCL activities than touch alone. The results suggest that the positive emotional effects of robotic touch can be boosted by combining elements of speech.

DFusion: Denoised TSDF Fusion of Multiple Depth Maps with Sensor Pose Noises.

The truncated signed distance function (TSDF) fusion is one of the key operations in the 3D reconstruction process. However, existing TSDF fusion methods usually suffer from the inevitable sensor noises. In this paper, we propose a new TSDF fusion network, named DFusion, to minimize the influences from the two most common sensor noises, i.e., depth noises and pose noises. To the best of our knowledge, this is the first depth fusion for resolving both depth noises and pose noises. DFusion consists of a fusion module, which fuses depth maps together and generates a TSDF volume, as well as the following denoising module, which takes the TSDF volume as the input and removes both depth noises and pose noises. To utilize the 3D structural information of the TSDF volume, 3D convolutional layers are used in the encoder and decoder parts of the denoising module. In addition, a specially-designed loss function is adopted to improve the fusion performance in object and surface regions. The experiments are conducted on a synthetic dataset as well as a real-scene dataset. The results prove that our method outperforms existing methods.

Surface Remeshing: A Systematic Literature Review of Methods and Research Directions.

Triangle meshes are used in many important shape-related applications including geometric modeling, animation production, system simulation, and visualization. However, these meshes are typically generated in raw form with several defects and poor-quality elements, obstructing them from practical application. Over the past decades, different surface remeshing techniques have been presented to improve these poor-quality meshes prior to the downstream utilization. A typical surface remeshing algorithm converts an input mesh into a higher quality mesh with consideration of given quality requirements as well as an acceptable approximation to the input mesh. In recent years, surface remeshing has gained significant attention from researchers and engineers, and several remeshing algorithms have been proposed. However, there has been no survey article on remeshing methods in general with a defined search strategy and article selection mechanism covering the recent approaches in surface remeshing domain with a good connection to classical approaches. In this article, we present a survey on surface remeshing techniques, classifying all collected articles in different categories and analyzing specific methods with their advantages, disadvantages, and possible future improvements. Following the systematic literature review methodology, we define step-by-step guidelines throughout the review process, including search strategy, literature inclusion/exclusion criteria, article quality assessment, and data extraction. With the aim of literature collection and classification based on data extraction, we summarized collected articles, considering the key remeshing objectives, the way the mesh quality is defined and improved, and the way their techniques are compared with other previous methods. Remeshing objectives are described by angle range control, feature preservation, error control, valence optimization, and remeshing compatibility. The metrics used in the literature for the evaluation of surface remeshing algorithms are discussed. Meshing techniques are compared with other related methods via a comprehensive table with indices of the method name, the remeshing challenge met and solved, the category the method belongs to, and the year of publication. We expect this survey to be a practical reference for surface remeshing in terms of literature classification, method analysis, and future prospects.

Positive Emotion Amplification by Representing Excitement Scene with TV Chat Agents.

This paper proposes emotion amplification for TV chat agents allowing users to get more excited in TV sports programs, and a model that estimates the excitement level of TV programs based on the number of social comment posts. The proposed model extracts the exciting intervals from social comments to the program scenes. By synchronizing recorded video streams and the intervals, the agents may talk with the user dynamically changing the frequency and volume of upbeat utterances, increasing the excitement of the user. To test these agents, participants watched TV content under three conditions: without an agent, with four agents that utter with a flat voice, and with four agents with emotion amplification. Results from 24 young adult Japanese individuals showed that their arousal of participants' subjective and physiological emotional responses were boosted because of the agents, enhancing their motivation to interact with the agent in the future. With empirical evidence, this paper supports these expectations and demonstrates that these agents can amplify the positive emotions of TV watchers, enhancing their motivation to interact with the agent in the future.

Decomposition procedure using methyl orthoformate to analyze silicone polymers.

A new decomposition method for structural analysis of polysiloxanes (silicones) was developed using methyl orthoformate. The siloxane bonds in samples with vinyl and/or methyl side groups decomposed under relatively mild acidic conditions up to around 70°C and were followed by methoxylation at the cleaved linkages with few side reactions. The product yields with respect to the siloxane monomer units were 98-100% for low molecular weight model siloxane compounds. Additionally, this method decomposed the silicone polymer sample in a similar manner with decomposition yields of 98 and 103% for the dimethylsiloxane main chain and dimethylvinylsilyl end groups, respectively. These results demonstrate that the proposed decomposition method should be an effective pretreatment procedure for structural and compositional analyses of silicone polymers.

Geometrically-correct projection-based texture mapping onto a deformable object.

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object’s surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems.