Effect of Normal Force Intensity on Tactile Motion Speed Perception Based on Spatiotemporal Cue.

While the relative motion between the skin and objects in contact with it is essential to everyday tactile experiences, our understanding of how tactile motion is perceived via human tactile function is limited. Previous studies have explored the effect of normal force on speed perception under conditions where multiple motion cues on the skin (spatiotemporal cue, tangential skin deformation cue, and slip-induced vibration cue) were integrated. However, the effect of the normal force on speed perception in terms of each motion cue remains unclear since the multiple motion cues have not been adequately separated in the previously reported experiments. In this study, we aim to elucidate the effect of normal force in situations where the speed perception of tactile motion is based solely on a spatiotemporal cue. We developed a pin-array display which allowed us to vary the intensity of the normal force without causing tangential forces or slip-induced vibrations. Using the display, we conducted two psychophysical experiments. In Experiment 1, we found that the speed of the object was perceived to be 1.12-1.14 times faster when the intensity of the normal force was doubled. In Experiment 2, we did not observe significant differences in the discriminability of tactile speed caused by differences in normal force intensity. Our experimental results are of scientific significance and offer insights for engineering applications when using haptic displays that can only provide spatiotemporal cues represented by normal forces.

Elucidating Diurnal Patterns in Touch Desire Using Social Media Data Toward Design of Haptic Applications and Displays.

Advances in haptic technology have led researchers and engineers to seek out killer applications in which users can enjoy an experience of touch in AR/VR spaces. Such applications will respond appropriately to human desire for haptic experiences (i.e., touch desire) and thus it is essential for researchers and engineers to understand the nature of people's touch desires as they arise in the course of daily life. In this study, we employed Twitter data analysis to investigate a diurnal pattern in touch desire. Our results showed that touch desire identified in and extracted from Twitter texts did reveal a diurnal pattern. Touch desire tended to be at its lowest in the morning and increased as the day progressed. The time at which it peaked varied with the specific target of touch desire. Touch desire in relation to other people and objects reached its peak at night, but touch desire in relation to animals reached its peak at noon. These results were confirmed not only by our Twitter text analysis but also by data from other social media and an online survey. In addition, we found that the diurnal pattern of touch desire for each target shows a strong correlation with that of visual desire for the same target. This suggests that the diurnal pattern of touch desire is not limited to the sense of touch but is common to other sensory desires for each target. Our findings suggest that researchers need to take the time of day into account when investigating touch desire. Our findings also offer valuable insights for developers into the design of haptic applications and displays that takes into account the timing of daily peaks in touch desire.

Toward Designing Haptic Displays for Desired Touch Targets: A Study of User Expectation for Haptic Properties via Crowdsourcing.

Things that people desire to touch in daily life are known to be limited to a number of specific targets (e.g., cats). The utilization of haptic displays to provide the experience of touching such desired targets is expected to enhance people's quality of life. However, it is currently unclear which haptic properties (e.g., hardness and weight) of desired targets should be rendered with haptic displays, and how they should be rendered. To address these issues, we conducted an experiment with 600 Japanese participants via crowdsourcing. Among the 600 participants, we identified potential users of haptic displays and analyzed their responses for each target. For each desired target, we identified the haptic properties in relation to which a "need for consistency" was felt by potential users between their expectations and actual impressions during touching. We also identified the haptic properties in relation to which a "biased impression" was held by potential users for each target. For example, potential users responded that cats were soft and that the actual impression of softness during touching needed to be consistent with their impression. Our results provide insights into the design of haptic displays for realizing desired touch experiences.

Age and Gender Differences in the Pseudo-Haptic Effect on Computer Mouse Operation in a Desktop Environment.

Pseudo-haptics is a method that can provide a haptic sensation without requiring a physical haptic device. The effect of pseudo-haptics is known to depend on the individual, but it is unclear which factors cause individual differences. As the first study establishing a calibration method for these differences in future research, we examined the differences in the pseudo-haptic effect on mouse cursor operation in a desktop environment depending on the age and gender of the user. We conducted an online experiment and collected data from more than 400 participants. The participants performed a task of lifting a virtual object with a mouse pointer. We found that the effect of pseudo-haptics was greater in younger or male participants than in older or female participants. We also found that the effect of pseudo-haptics, which varied with age and gender, can be explained by habituation to the mouse in daily life and the accuracy of detecting the pointer position using vision or proprioception. Specifically, the pseudo-haptic effect was higher for those who used the mouse more frequently and had higher accuracy in identifying the pointer position using proprioception or vision. The results of the present study not only indicate the factors that cause age and gender differences but also provide hints for calibrating these differences.

Pseudo-Haptic Heaviness Influenced by the Range of the C/D Ratio and the Position of the C/D Ratio Within a Given Range.

Pseudo-haptic heaviness refers to the illusory sensation of heaviness caused by a dissociation in amplitudes between object movements on a screen and users' motor actions. The amplitude ratio of object movements to the user's actions, the so-called C/D ratio, is a powerful determinant of pseudo-haptic heaviness. According to previous studies, perceptual judgments for a given stimulus value are influenced by the position of the value within a given stimulus range, while no studies have shown the same to be true for pseudo-haptic heaviness. The present study examined whether pseudo-haptic heaviness determined by the C/D ratio was influenced by the range of C/D ratios, and also, by the position of the C/D ratio within a given range. Participants were asked to drag and lift a square on the screen up to a target line and then rate its heaviness; the range of C/D ratios was controlled as a between-participants factor. We observed a phenomenon whereby both the range and position of the C/D ratio influenced the rated heaviness. This phenomenon was clearly established over 8 experimental trials. We conclude that both the C/D ratio range and the position of the C/D ratio within a given range are determinants for pseudo-haptic heaviness.

Softness Perception of Visual Objects Controlled by Touchless Inputs: The Role of Effective Distance of Hand Movements.

Feedback on the material properties of a visual object is essential in enhancing the users' perceptual experience of the object when users control the object with touchless inputs. Focusing on the softness perception of the object, we examined how the effective distance of hand movements influenced the degree of the object's softness perceived by users. In the experiments, participants moved their right hand in front of a camera which tracked their hand position. A textured 2D or 3D object on display deformed depending on the participant's hand position. In addition to establishing a ratio of deformation magnitude to the distance of hand movements, we altered the effective distance of hand movement, within which the hand movement could deform the object. Participants rated the strength of perceived softness (Experiments 1 and 2) and other perceptual impressions (Experiment 3). A longer effective distance produced a softer impression of the 2D and 3D objects. The saturation speed of object deformation due to the effective distance was not a critical determinant. The effective distance also modulated other perceptual impressions than softness. The role of the effective distance of hand movements on perceptual impressions of objects under touchless control is discussed.

Falling and heaviness: Heaviness judgment for a visual object which users lift up is influenced by the presentation of the object's falling or staying still.

When lifting and subsequently releasing a visual object on a screen using a computer mouse, users tend to judge the object to be heavier when the motion speed of the object during lifting is smaller. However it was unclear how the presentation of an object falling after its release influenced the judgment of heaviness. Users generally believe mistakenly that heavier objects fall faster. Based on the previous report of this misbelief, we briefly explored how the falling speed of a visual object after release by a user influenced the judgment of heaviness. The falling speed of the object was systematically modulated by changing gravity in the simulation of the natural falling of the object. Participants judged the object's heaviness after they lifted and subsequently released it. As a result, the participants judged the object to be lighter when the falling speed was zero. However, no significant differences were observed among the conditions with a falling speed greater than zero. It is suggested that for the judgment of heaviness, a vital aspect in the presentation of a falling object after releasing is whether the object falls or not.

Force illusion induced by visual illusion: Illusory curve in cursor path is interpreted as unintended force.

Discrepancies between expected and actual visual outcomes of motor action can produce an illusory sensation of unintended force. In the present study, we addressed whether the force illusion could be induced even when the discrepancy was brought about by the illusory appearance of the actual outcome. Specifically, the apparent path of a cursor controlled by the participants was modulated by the direction of noise motion presented inside the cursor. We showed that a greater noise motion inside the cursor caused a greater apparent curve of the cursor path and, also, higher rating scores for an unintended force. We also found that the unintended force was influenced strongly by the visibility of the cursor, suggesting that the apparent curve of the cursor path was a more important factor in generating the unintended force than the noise motion itself inside the cursor. Our results suggest that the illusory force, which is mediated by cross-modal mechanisms susceptible to visual illusion, can be exploited in extended reality systems as a novel technique for giving users a sensation of force.

Specifying Visual Parameters for Haptic-Visual Sequential Matching of Material Softness.

When shopping online, a customer cannot directly touch the products but may sometimes make judgments about the haptic properties of a product based only on visual information, before making a purchase decision. In this scenario, a customer may be dissatisfied if there is an inconsistency in the judgment of the product's haptic properties they made before purchasing, and their actual experience of those haptic properties once they have received the product. Thus, it is necessary for online sellers to appropriately optimize visual information for materials so that perceived softness is consistent between haptic and visual modalities presented in different locations and at different moments in time. Focusing on visual indentation depth and speed, we examined the visual parameters used to sequentially match haptic and visual softness from haptic and visual information made available in different locations and at different times. Participants performed a two-alternative forced choice task to determine which of two video clips contained an elastic material with a softness impression most similar to the haptic softness of an actual material that the participants indented with their index finger. Based on a sequence of 25 repeated judgments for each material, our algorithm optimized each visual parameter based on a Gaussian process. The optimized visual indentation depth varied consistently with material compliance, while the optimized visual indentation speed did not, suggesting that visual indentation depth was critical for softness matching. The optimized visual indentation depth was highly correlated with the haptic indentation depth. Subjective rating scores for the softness matching increased significantly after the optimization process. The results indicate that participants are able to successfully match the haptic and visual softness of materials by checking the relationship between indentation depths detected haptically, and those detected visually.

Tracking changes in touch desire and touch avoidance before and after the COVID-19 outbreak.

Touch is essential for survival, social bonding, and overall health. However, the COVID-19 pandemic calls for an abrupt withdrawal from physical contact, and the prolonged lockdown has left many people in solitude without touch for months. This unprecedented dissociation from touch has cast a shadow on people's mental and physical well-being. Here we approached the issue by examining COVID-19's impact on people's touch attitudes. We analyzed people's desire and avoidance for animate and inanimate targets based on large-scale Japanese Twitter posts over an 8-year span. We analyzed the impact of the COVID-19 outbreak with the difference-in-differences estimation method, which can estimate the impact while accounting for other changes over time such as seasonality or long-term effects. As a result, we found that people's desire for touching the human body and pet animals increased significantly after the COVID-19 outbreak and remained high afterward. In contrast, the avoidance of touching everyday objects (e.g., doorknobs and money) increased immediately after the outbreak but gradually returned to the pre-COVID-19 levels. Our findings manifest the impact of COVID-19 on human touch behavior. Most importantly, they highlight the sign of "skin hunger," a public health crisis due to social distancing, and call attention to the trend that people are becoming less aware of infection control as COVID-19 persists.

Vibrotactile Spatiotemporal Pattern Recognition in Two-Dimensional Space Around Hand.

This study investigated vibrotactile spatiotemporal pattern recognition in the two-dimensional space around a hand. The participants placed their hands on the medium, and identified the recognized pattern presented in the medium. There were 64 rotational patterns, presented with sequential impulse vibrations. We investigated how well humans recognized the patterns presented around their hand, and identified the pattern factors (e.g., rotational direction) that affected recognition accuracy. The probability of obtaining correct answers was 48.9 %. It was observed that the start and end points, rotational direction, and the number of vibrations affected recognition accuracy. It was also found that patterns starting or ending on the ulnar side ( 0°) of the hand were difficult to recognize, whereas those starting or ending on the distal ( 90°) or proximal side ( 270°) of the hand were easily recognizable. Furthermore, we found a type of the oblique effect. Patterns starting or ending in the oblique direction were more difficult to recognize than those in the cardinal direction. We also found that the clockwise rotational pattern was slightly easier to recognize than the counterclockwise rotational pattern. Finally, the underestimation of the judgment of tactile numerosity explains how the number of vibrations in the patterns affected the recognition accuracy. This result can be used as a baseline when the recognition of spatiotemporal patterns outside the body under other conditions is examined in future studies.

Underestimation in temporal numerosity judgments computationally explained by population coding model.

The ability to judge numerosity is essential to an animal's survival. Nevertheless, the number of signals presented in a sequence is often underestimated. We attempted to elucidate the mechanism for the underestimation by means of computational modeling based on population coding. In the model, the population of neurons which were selective to the logarithmic number of signals responded to sequential signals and the population activity was integrated by a temporal window. The total number of signals was decoded by a weighted average of the integrated activity. The model predicted well the general trends in the human data while the prediction was not fully sufficient for the novel aging effect wherein underestimation was significantly greater for the elderly than for the young in specific stimulus conditions. Barring the aging effect, we can conclude that humans judge the number of signals in sequence by temporally integrating the neural representations of numerosity.

Sinusoidal Vibration Source Localization in Two-Dimensional Space Around the Hand.

There are use cases where presenting spatial information via the tactile sense is useful (e.g., situations where visual and audio senses are not available). Conventional methods that directly attach a vibrotactile array to a user's body present spatial information such as direction by having users localize the vibration source from among the sources in the array. These methods suffer from problems such as heat generation of the actuator or the installation cost of the actuators in a limited space. A promising method of coping with these problems is to place the vibrotactile array at a distance from the body, instead of directly attaching it to the body, with the aim of presenting spatial information in the same way as the conventional method. The present study investigates the method's effectiveness by means of a psychophysical experiment. Specifically, we presented users with sinusoidal vibrations from remote vibrotactile arrays in the space around the hand and asked them to localize the source of the vibration. We conducted an experiment to investigate the localization ability by using two vibration frequencies (30 Hz as a low frequency and 230 Hz as a high frequency). We chose these two frequencies since they effectively activate two distinctive vibrotactile channels: the rapidly adapting afferent channel and the Pacinian channel. The experimental results showed that humans can recognize the direction of the vibration source, but not the distance, regardless of the source frequency. The accuracy of the direction recognition varied slightly according to the vibration source direction, and also according to the vibration frequency. This suggests that the calibration of stimulus direction is required in the case of both high and low frequencies for presenting direction accurately as intended. In addition, the accuracy variance of direction recognition increased as the source became farther away, and the degree of increase was especially large with the low-frequency source. This suggests that a high frequency is recommended for presenting accurate direction with low variance.

The Relationship Between Illusory Heaviness Sensation and the Motion Speed of Visual Feedback in Gesture-Based Touchless Inputs.

Interaction systems with gesture-based touchless inputs are becoming more common. Nevertheless, perceptual properties of the visual feedback used in the system have not been well documented. We investigated whether the speed of motion shown in visual feedback used in gesture-based touchless inputs could be a cue for the heaviness sensation of an object even when other incidental cues, such as changes in object size and spatial consistencies in direction between gestures and feedback, were eliminated from the stimuli. Participants were asked to make a gesture to grasp and raise/lower disks shown on a horizontal display. The disk's diameter changed in accordance with the vertical position of the participant's hand. The results showed that the rate of change in diameter determined the heaviness sensation. When the disks were replaced with concentric gratings having sinusoidal radial intensity and thus the cue of size change was eliminated from the stimuli, the heaviness sensation was dependent on the speed of phase shift (that is, motion) in the grating. It was also found that spatial consistency between the direction of gestures and phase shift was not a critical condition for the heaviness sensation. Finally, the speed of motion served as a critical determinant of the heaviness sensation even when another visual feature (i.e., frame rate) was modulated in a single session, which indicates that the effect of the speed of motion on the heaviness sensation was unlikely due to demanded characteristics. The results indicate that the heaviness sensation for visual feedback of gesture-based touchless inputs is based purely on the speed of the visual feedback motion.

Visual estimation of the force applied by another person.

As observers, we believe that we can visually estimate the force that another person is applying to a material. However, it is unclear what kind of cues we use to do this. We focused on two types of visual change that occur when actors push an elastic material from above with their fingers: visual shaking and visual indentation depth. The first one relates to a finger/hand shaking, known as an "induced tremor", and the second one relates to material deformation due to the application of force. We found that human observers mainly used visual shaking to estimate the force being applied by another person in a video clip. Overall, the apparent applied force was perceived to be stronger when the level of visual shaking was greater. We also found that observers mainly used visual indentation depth and visual shaking to estimate the softness rating of materials. Overall, the apparent softness was perceived to be greater when the visual indentation depth was larger and the level of visual shaking was lower, which indicates that observers use visual shaking to estimate the force being applied, and that estimated force is then used for an estimation of softness.

Deformation Matching: Force Computation Based on Deformation Optimization.

The tactile information to be presented to a user during interaction with a virtual object is calculated by simulating the contact between the object model and user model. In the simulation, a distributed force is applied to the contact area on the skin tissue of users' hands and results in deformation of the skin tissue. The skin deformation caused by the distributed force is the target contact state that should be presented by the device. However, most multipoint haptic displays do not have sufficient degrees of freedom (DoF) to represent the target contact state. This paper presents the concept and formulation of "deformation matching," whereby the output force is calculated to minimize the error between the target skin deformation and skin deformation that can be realized by the limited DoF device's output force. For comparison, the conventional concept of "force matching" was also formulated. The difference in human perception between these two concepts in the expression of friction was investigated through experiments using a pin-array tactile display capable of stimulating 128 points. It was demonstrated that the perception of the friction coefficient was more sensitive and the perception of the friction direction was more accurate in deformation matching than in force matching.

Perceptual judgments for the softness of materials under indentation.

Humans can judge the softness of elastic materials through only visual cues. However, factors contributing to the judgment of visual softness are not yet fully understood. We conducted a psychophysical experiment to determine which factors and motion features contribute to the apparent softness of materials. Observers watched video clips in which materials were indented from the top surface to a certain depth, and reported the apparent softness of the materials. The depth and speed of indentation were systematically manipulated. As physical characteristics of materials, compliance was also controlled. It was found that higher indentation speeds resulted in larger softness rating scores and the variation with the indentation speed was successfully explained by the image motion speed. The indentation depth had a powerful effect on the softness rating scores and the variation with the indentation depth was consistently explained by motion features related to overall deformation. Higher material compliance resulted in higher softness rating scores and these variation with the material compliance can be explained also by overall deformation. We conclude that the brain makes visual judgments about the softness of materials under indentation on the basis of the motion speed and deformation magnitude.

Getting Insights From Twitter: What People Want to Touch in Daily Life.

Understanding what people want to touch in daily life has been one of the central topics in the fields of haptic science, engineering, and marketing. Several studies have addressed the topic, however, their findings were highly dependent on the experimental stimuli in the laboratory environment. In this study, we tried to gain insights into targets that people want to touch in daily life by conducting a Twitter survey. We collected a considerable amount of Japanese Twitter posts containing references to the desire for touch. To capture the motivation underlying these desires in relation to haptic properties, we used several queries that comprehensively covered exploratory procedures. The results showed that targets that people wanted to touch depended on the exploratory procedures in the queries used (e.g., "want to stroke" tended to target people and animals). We also found that these targets were desirable to touch not only for their haptic properties but also as a means of communicating with other people or living animals. Our findings would be important for understanding human haptic function in the real world and for developing consumer haptic displays and applications.

Impact Vibration Source Localization in Two-Dimensional Space Around Hand.

This article investigated the localization ability of an impulse vibration source outside the body in two-dimensional space. We tested whether humans can recognize the direction or distance of an impulse vibration source when using their hand to detect spatiotemporal vibrotactile information provided by the propagated vibrational wave from the source. Specifically, we had users put their hands on a silicone rubber sheet in several postures. We asked users to indicate the position of the vibration source when a location on the sheet was indented. Experimental results suggested that the direction of the impact vibration source can be recognized to some extent, although recognition accuracy depends on hand posture and the position of the vibration source. The best results were achieved when the fingers and palm were grounded and a vibration source was presented around the middle fingertip, and the directional recognition error in this case was 6 °. In contrast, results suggest it is difficult to accurately recognize the distance of the vibration. The results of this study suggest a new possibility for directional display where vibrotactile actuators are embedded at a distance from the user's hand.

Sense of Resistance for a Cursor Moved by User's Keystrokes.

Haptic sensation of a material can be modulated by its visual appearance. A technique that utilizes this visual-haptic interaction is called as pseudo-haptic feedback. Conventional studies have investigated pseudo-haptic feedback in situations, wherein a user manipulated a virtual object using a computer mouse, a force-feedback device, etc. The present study investigated whether and how it was possible to offer pseudo-haptic feedback to a user who manipulated a virtual object using keystrokes. Participants moved a cursor toward a destination by pressing a key. While the cursor was moving, the cursor was temporarily slowed down on a square area of the screen. The participants' task was to report, on a five-point scale, how much resistance they felt to the cursor's movement. In addition to the basic speed of the cursor, the ratio of the basic speed to the speed within the square area was varied. In Experiment 1, we found that these two factors interacted significantly with each other, but further analysis showed that the cursor speed within the square area was the most important determinant of perceived resistance. In Experiment 2, consistent with the results of the previous experiment, it was found that the cursor movement outside of the square area was not required to generate the sense of resistance. Counterintuitively, in Experiment 3, the sense of resistance was apparent even without user's keystrokes. We discuss how the sense of resistance for a cursor moved by keystrokes can be triggered visually, but interpreted by the brain as a haptic impression.