Egocentric Boundaries on Distinguishing Colliding and Non-Colliding Pedestrians while Walking in a Virtual Environment.

Avoiding person-to-person collisions is critical for visual field loss patients. Any intervention claiming to improve the safety of such patients should empirically demonstrate its efficacy. To design a VR mobility testing platform presenting multiple pedestrians, a distinction between colliding and non-colliding pedestrians must be clearly defined. We measured nine normally sighted subjects' collision envelopes (CE; an egocentric boundary distinguishing collision and non-collision) and found it changes based on the approaching pedestrian's bearing angle and speed. For person-to-person collision events for the VR mobility testing platform, non-colliding pedestrians should not evade the CE.

Gaze Scanning at Street Crossings by Pedestrians With Homonymous Hemianopia With and Without Hemispatial Neglect.

Purpose: To investigate compensatory gaze-scanning behaviors during street crossings by pedestrians with homonymous hemianopia (HH) and hemispatial neglect (HSN). Methods: Pedestrians with right homonymous hemianopia (RHH) and left homonymous hemianopia without (LHH) and with left spatial-neglect (LHSN) walked on city streets wearing a gaze-tracking system that also captured scene videos. Street-crossing instances were manually annotated, and horizontal gaze scan of magnitude ≥20° and scanning rates were compared within-subject, between the side of the hemifield loss (BlindSide) and the other side (SeeingSide). Proportion of instances with scans to both the left and the right side at nonsignalized crossings (indicative of safe scanning behavior) were compared among the three subject groups. Results: Data from 19 participants (6 LHH, 7 RHH, and 6 with mild [4] or moderate [2] LHSN), consisting of 521 street-crossing instances of a total duration of 201 minutes and 5375 gaze scans, were analyzed. The overall gaze magnitude (mean [95% confidence interval (CI)]) was significantly larger toward the BlindSide (40.4° [39.1°-41.9°]) than the SeeingSide (36° [34.8°-37.3°]; P < 0.001). The scanning rate (mean [95% CI] scans/min) toward the BlindSide (14 [12.5-15.6]) was significantly higher than the SeeingSide (11.5 [10.3°-12.9°]; P < 0.001). The scanning rate in the LHSN group (10.7 [8.9-12.8]) was significantly lower than the LHH group (14 [11.6-17.0]; P = 0.045). The proportion of nonsignalized crossings with scans to both sides was significantly lower in LHSN (58%; P = 0.039) and RHH (51%; P = 0.003) than LHH (75%) participants. Conclusions: All groups demonstrated compensatory scanning, making more gaze scans with larger magnitudes to the blind side. Mild to moderate LHSN adversely impacted the scanning rate.

The Invisibility of Scotomas I: The Carving Hypothesis.

SIGNIFICANCE: Veridical depictions of scene appearance with scotomas allow better understanding of the impact of field loss and may improve the development and implementation of rehabilitation. Explanation and depiction of the invisibility of scotoma may lead to patients' understanding and thus better compliance with related treatments. PURPOSE: Simulations of perception with scotomas guide training, patient education, and rehabilitation research. Most simulations incorrectly depict scotomas as black patches, although the scotomas and the missing contents are usually invisible to patients. We present a novel approach to capture the reported appearance of scenes with scotomas. METHODS: We applied a content-aware image resizing algorithm to carve out the content elided under the scotomas. With video sequences, we show how and why eye movements fail to increase the visibility of the carved scotomas. RESULTS: Numerous effects, reported by patients, emerge naturally from the scotoma carving. Carving-eliminated scotomas over natural images are barely visible, despite causing substantial distortions. Low resolution and contrast sensitivity at farther eccentricities and saccadic blur reduce the visibility of the distortions. In a walking scenario, static objects moving smoothly to the periphery disappear into and then reemerge out of peripheral scotomas, invisibly. CONCLUSIONS: Scotoma carving provides a viable hypothetical simulation of vision with scotomas due to loss of neurons at the retinal ganglion cell level and higher. As a hypothesis, it generates predictions that lend themselves to future clinical testing. The different effects of scotomas due to loss of photoreceptors are left for follow-up work.

Oblique multi-periscopic prism for field expansion of homonymous hemianopia.

Oblique Fresnel peripheral prisms have been used for field expansion in homonymous hemianopia mobility such as walking and driving. However, limited field expansion, low image quality, and small eye scanning range limit their effectiveness. We developed a new oblique multi-periscopic prism using a cascade of rotated half-penta prisms, which provides 42° horizontal field expansion along with 18° vertical shift, high image quality, and wider eye scanning range. Feasibility and performance of a prototype using 3D-printed module are demonstrated by raytracing, photographic depiction, and Goldmann perimetry with patients with homonymous hemianopia.

Monovision Correction Preference and Eye Dominance Measurements.

Purpose: To propose new methods for eye selection in presbyopic monovision corrections. Methods: Twenty subjects with presbyopia performed two standard methods of binary eye dominance identification (sensory with +1.50 diopters [D ]and +0.50 D and sighting with "hole-in-the-card") and two psychophysical methods of perceived visual quality: (1) the Preferential test, 26 natural images were judged with the near addition in one eye or in the other in a 2-interval forced-choice task, and the Eye Dominance Strength (EDS) defined as the proportion of trials where one monovision is preferred over the other; (2) the Multifocal Acceptance Score (MAS-2EV) test, the perceived quality of a natural images set (for 2 luminance levels and distances) was scored and EDS defined as the score difference between monovision in one eye or the other. Left-eye and right-eye dominance are indicated with negative and positive values, respectively. Tests were performed using a Simultaneous Vision Simulator, which allows rapid changes between corrections. Results: Standard sensory and sighting dominances matched in only 55% of subjects. The Preferential EDS (ranging from -0.7 to +0.9) and MAS-2EV EDS (ranging from -0.6 to +0.4) were highly correlated. Selecting the eye for far in monovision with the MAS-2EV, sensory, or sighting tests would have resulted in 79%, 64%, and 43% success considering the Preferential test as the gold standard. Conclusions: Tests based on perceptual preference allow selection of the preferred monovision correction and measurement of dominance strength. Translational Relevance: The binocular visual simulator allows efficient implementation of eye preference tests for monovision in clinical use.

Development of Virtual Reality Walking Collision Detection Test on Head-mounted display.

Detecting and avoiding collisions during walking is critical for safe mobility. To determine the effectiveness of clinical interventions, a realistic objective outcome measure is needed. A real-world obstacle course with moving hazards has numerous limitations (e.g., safety concerns of physical collision, inability to control events, maintaining event consistency, and event randomization). Virtual reality (VR) platforms may overcome such limitations. We developed a VR walking collision detection test using a standalone head-mounted display (HMD, Meta Quest 2) with the Unity 3D engine to enable subjects' physical walking within a VR environment (i.e., a busy shopping mall). The performance measures focus on the detection and avoidance of potential collisions, where a pedestrian may (or may not) walks toward a collision with the subject, while various non-colliding pedestrians are presented simultaneously. The physical space required for the system was minimized. During the development, we addressed expected and unexpected hurdles, such as mismatch of visual perception of VR space, limited field of view (FOV) afforded by the HMD, design of pedestrian paths, design of the subject task, handling of subject's response (detection or avoidance behavior), use of mixed reality (MR) for walking path calibration. We report the initial implementation of the HMD VR walking collision detection and avoidance scenarios that showed promising potential as clinical outcome measures.

Comparison of visual SLAM and IMU in tracking head movement outdoors.

Tracking head movement in outdoor activities is more challenging than in controlled indoor lab environments. Large-magnitude head scanning is common under natural conditions. Compensatory gaze (head and eye) scanning while walking may be critical for people with visual field loss. We compared the accuracy of two outdoor head tracking methods: differential inertial measurement units (IMU) and simultaneous localization and mapping (SLAM). At a fixed location experiment, a gaze aiming test showed that SLAM outperforms IMU in terms of error (IMU: 9.6°, SLAM: 4.47°). In an urban street walking experiment conducted with five patients with hemifield loss, the IMU drift, quantified by root-mean-square deviation, was as high as 68.1°, while the drift of SLAM was only 5.3°. However, the SLAM method suffered from data loss due to tracking failure (~10% overall, and ~ 18% when crossing streets). Our results show that the SLAM and IMU methods have complementary properties. Because of no data gaps, the differential IMU method may be desirable as compared to SLAM in settings where the signal drift can be removed in post-processing and small gaze estimation errors can be tolerated.

Beyond the Cane: Describing Urban Scenes to Blind People for Mobility Tasks.

Blind people face difficulties with independent mobility, impacting employment prospects, social inclusion, and quality of life. Given the advancements in computer vision, with more efficient and effective automated information extraction from visual scenes, it is important to determine what information is worth conveying to blind travelers, especially since people have a limited capacity to receive and process sensory information. We aimed to investigate which objects in a street scene are useful to describe and how those objects should be described. Thirteen cane-using participants, five of whom were early blind, took part in two urban walking experiments. In the first experiment, participants were asked to voice their information needs in the form of questions to the experimenter. In the second experiment, participants were asked to score scene descriptions and navigation instructions, provided by the experimenter, in terms of their usefulness. The descriptions included a variety of objects with various annotations per object. Additionally, we asked participants to rank order the objects and the different descriptions per object in terms of priority and explain why the provided information is or is not useful to them. The results reveal differences between early and late blind participants. Late blind participants requested information more frequently and prioritized information about objects' locations. Our results illustrate how different factors, such as the level of detail, relative position, and what type of information is provided when describing an object, affected the usefulness of scene descriptions. Participants explained how they (indirectly) used information, but they were frequently unable to explain their ratings. The results distinguish between various types of travel information, underscore the importance of featuring these types at multiple levels of abstraction, and highlight gaps in current understanding of travel information needs. Elucidating the information needs of blind travelers is critical for the development of more useful assistive technologies.

Preparing participants for the use of the tongue visual sensory substitution device.

PURPOSE: Visual sensory substitution devices (SSDs) convey visual information to a blind person through another sensory modality. Using a visual SSD in various daily activities requires training prior to use the device independently. Yet, there is limited literature about procedures and outcomes of the training conducted for preparing users for practical use of SSDs in daily activities. METHODS: We trained 29 blind adults (9 with congenital and 20 with acquired blindness) in the use of a commercially available electro-tactile SSD, BrainPort. We describe a structured training protocol adapted from the previous studies, responses of participants, and we present retrospective qualitative data on the progress of participants during the training. RESULTS: The length of the training was not a critical factor in reaching an advanced stage. Though performance in the first two sessions seems to be a good indicator of participants' ability to progress in the training protocol, there are large individual differences in how far and how fast each participant can progress in the training protocol. There are differences between congenital blind users and those blinded later in life. CONCLUSIONS: The information on the training progression would be of interest to researchers preparing studies, and to eye care professionals, who may advise patients to use SSDs.IMPLICATIONS FOR REHABILITATIONThere are large individual differences in how far and how fast each participant can learn to use a visual-to-tactile sensory substitution device for a variety of tasks.Recognition is mainly achieved through top-down processing with prior knowledge about the possible responses. Therefore, the generalizability is still questionable.Users develop different strategies in order to succeed in training tasks.

Effects of Perceptual-motor Training on Collision Judgments with Peripheral Prism Expanded Vision.

SIGNIFICANCE: Peripheral prisms (p-prisms) improve blind-side detection of hazards in hemianopia by shifting the image of the hazard into the intact visual field. Collision judgments can be made accurately after detection by using a gaze shift to fixate the hazard in the prism-free portion of the lens, but this is slow relative to normal peripheral vision. A prior study found that prism adaptation for visual direction did not occur with general wear. We developed a perceptual-motor training regimen that resulted in accurate pointing at p-prism targets after six 1-hour sessions. PURPOSE: This study aimed to determine if improvements in pointing accuracy from perceptual-motor training generalized to collision judgments during simulated walking. METHODS: Participants with hemianopia (n = 13) made collision judgments in virtual reality for a person appearing 0.4 to 13.5° from the walking path. Judgments were measured under fixed gaze, requiring collision judgments via the p-prism image only, and free gaze, representing a more natural scenario. Measurements were made without and with p-prisms immediately after fitting, after a 2-week acclimation, after training, and 3 months later. Controls (n = 13) did one visit without p-prisms. RESULTS: Controls had 100% detection and symmetrically distributed collision judgments for the central 33 and 36% of hazards under fixed gaze and free gaze, respectively. In hemianopia, the seeing side was not different from controls. Blind-side detection was reduced without p-prisms to 40% fixed gaze and 82% free gaze and improved with p-prisms to 99% fixed gaze and 97% free gaze (P < .001). When first worn, fixed-gaze prism side collisions were 63 versus 37% on the seeing side and 41 versus 39% for free gaze (P < .001). There was a small improvement for fixed gaze after the 2-week acclimation (53%, P < .001), but no improvements from training or an additional 3 months of use. CONCLUSIONS: P-prisms improved detection, but collision judgments were inaccurate when seen only via the p-prisms and did not improve with perceptual-motor training. Patients should continue to be advised to turn their head and eyes to fixate the hazard after detection.

Photographic Depiction of the Field of View with Spectacles-mounted Low Vision Aids.

SIGNIFICANCE: Photographic depiction helps to illustrate the primary and secondary field of view effects of low vision devices along with their utility to clinicians, patients, and caretakers. This technique may also be helpful for designers and researchers in improving the design and fitting of low vision devices. PURPOSE: The field of view through spectacles-mounted low vision devices has typically been evaluated using perimetry. However, the perimetric field diagram is different from the retinal image and often fails to represent the important aspects of the field of view and visual parameters. We developed a photographic depiction method to record and veridically show the field of view effects of these devices. METHODS: We used a 3D-printed holder to place spectacles-mounted devices at the same distance from the empirically determined reference point of the field of view in a camera lens (f = 16 mm) as they would be from an eye, when in use. The field of view effects of a bioptic telescope, a minifier (reverse telescope), and peripheral prisms were captured using a conventional camera, representing retinal images. The human eye pupil size (adjusting the F number: f/2.8 to f/8 and f/22 in the camera lens) and fitting parameters (pantoscopic tilt and back vertex distance) varied. RESULTS: Real-world indoor and outdoor walking and driving scenarios were depicted as retinal images illustrating the field of view through low vision devices, distinguishing optical and obscuration scotomas, and demonstrating secondary effects (spatial distortions, viewpoint changes, diplopia, spurious reflection, and multiplexing effects) not illustrated by perimetric field diagrams. CONCLUSIONS: Photographic depiction illustrates the primary and secondary field of view effects of the low vision devices. These images highlight the benefit and possible trade-offs of the low vision devices and may be beneficial in education and training.

Retinal prosthetic vision simulation: temporal aspects.

Objective. The perception of individuals fitted with retinal prostheses is not fully understood, although several retinal implants have been tested and commercialized. Realistic simulations of perception with retinal implants would be useful for future development and evaluation of such systems.Approach.We implemented a retinal prosthetic vision simulation, including temporal features, which have not been previously simulated. In particular, the simulation included temporal aspects such as persistence and perceptual fading of phosphenes and the electrode activation rate.Main results.The simulated phosphene persistence showed an effective reduction in flickering at low electrode activation rates. Although persistence has a positive effect on static scenes, it smears dynamic scenes. Perceptual fading following continuous stimulation affects prosthetic vision of both static and dynamic scenes by making them disappear completely or partially. However, we showed that perceptual fading of a static stimulus might be countered by head-scanning motions, which together with the persistence revealed the contours of the faded object. We also showed that changing the image polarity may improve simulated prosthetic vision in the presence of persistence and perceptual fading.Significance.Temporal aspects have important roles in prosthetic vision, as illustrated by the simulations. Considering these aspects may improve the future design, the training with, and evaluation of retinal prostheses.

Vision is protected against blue defocus.

Due to chromatic aberration, blue images are defocused when the eye is focused to the middle of the visible spectrum, yet we normally are not aware of chromatic blur. The eye suffers from monochromatic aberrations which degrade the optical quality of all images projected on the retina. The combination of monochromatic and chromatic aberrations is not additive and these aberrations may interact to improve image quality. Using Adaptive Optics, we investigated the optical and visual effects of correcting monochromatic aberrations when viewing polychromatic grayscale, green, and blue images. Correcting the eye's monochromatic aberrations improved optical quality of the focused green images and degraded the optical quality of defocused blue images, particularly in eyes with higher amounts of monochromatic aberrations. Perceptual judgments of image quality tracked the optical findings, but the perceptual impact of the monochromatic aberrations correction was smaller than the optical predictions. The visual system appears to be adapted to the blur produced by the native monochromatic aberrations, and possibly to defocus in blue.

Multi-periscopic prism device for field expansion.

Patients with visual field loss frequently collide with other pedestrians, with the highest risk being from pedestrians at a bearing angle of 45°. Current prismatic field expansion devices (≈30°) cannot cover pedestrians posing the highest risk and are limited by poor image quality and restricted eye scanning range (<5°). A new field expansion device: multi-periscopic prism (MPP); comprising a cascade of half-penta prisms provides wider shifting power (45°) with dramatically better image quality and wider eye scanning range (15°) is presented. Spectacles-mounted MPPs were implemented using 3D printing. The efficacy of the MPP is demonstrated using perimetry, photographic depiction, and analyses of the collision risk covered by the devices.

2017 Charles F. Prentice Award Lecture: Peripheral Prisms for Visual Field Expansion: A Translational Journey.

On the occasion of being awarded the Prentice Medal, I was asked to summarize my translational journey. Here I describe the process of becoming a low-vision rehabilitation clinician and researcher, frustrated by the unavailability of effective treatments for some conditions. This led to decades of working to understand patients' needs and the complexities and subtleties of their visual systems and conditions. It was followed by many iterations of developing vision aids and the techniques needed to objectively evaluate their benefit. I specifically address one path: the invention and development of peripheral prisms to expand the visual fields of patients with homonymous hemianopia, leading to our latest multiperiscopic prism (mirror-based design) with its clear 45° field-of-view image shift.

Structured Laser Light Improves Tripping Hazard Recognition for People with Visual Impairments.

Purpose: Using a geometrically derived model and a virtual curb simulator, we quantify the degree to which a wearable device that projects a laser line onto tripping hazards in a pedestrian's path improves visual recognition for people with visual impairments (VI). We confirm this with subjects' performance on computer simulations of low contrast curbs. Methods: We derive geometric expressions quantifying the visual cue users perceive when a single laser line is projected from their hip onto a curb. We show how the efficacy of this cue changes with the angle of the laser line relative to the subject's walking trajectory. We confirm this result with data from three subjects with VI in a simulated curb recognition task in which subjects classified computer images as an "Ascending," "Flat," or "Descending" curb. Results: The derived model predicts that human recognition performance depends strongly on the laser line angle and the subject data confirms this (r2 = 0.86,P < 0.001). The laser line cue improved subject accuracy from a chance level of 33% to 95% for a simulated, one-inch, low-contrast curb at a distance of five feet. Conclusions: Recognition of curbs in low light can be improved by augmenting the scene with a single laser line projected from a user's hip, if the angle of laser line is appropriately selected. Translational Relevance: A majority of people with VI rely on their impaired residual vision for mobility, rather than a mobility aid, resulting in increased injury for this population. Enhancing residual vision could promote safety, increase independence, and reduce medical costs.

Hazard Detection With Monocular Bioptic Telescopes in a Driving Simulator.

Purpose: In most states, people with reduced visual acuity may legally drive with the aid of a bioptic telescope. However, concerns have been raised that the ring scotoma may impair detection of peripheral hazards. Using a driving simulator, we tested the hypothesis that the fellow eye would be able to compensate for the ring scotoma when using a monocular telescope. Methods: Sixteen bioptic users completed three drives with binocular viewing interleaved between three drives with monocular viewing. Forty pedestrians appeared and ran on the road for 1 second, including 26 within the ring scotoma, while participants were reading road signs through their own monocular telescopes. Head movements were analyzed to determine whether the pedestrian appeared before or only while using the telescope. Results: For pedestrians that appeared only during bioptic use and were likely in the area of the ring scotoma, detection rates were significantly higher in binocular (fellow eye can compensate) than monocular (fellow eye patched) viewing (69% vs. 32%; P < 0.001); this was true for both current and noncurrent drivers. For pedestrians appearing before or after bioptic use, detection rates did not differ in binocular and monocular viewing. However, detection rates were even higher and reaction times shorter when the telescope was not being used. Conclusions: Both current and noncurrent drivers'  fellow eyes were able to compensate, at least in part, for the ring scotoma. Translational Relevance: When using monocular telescopes, the fellow eye reduces the impact of the ring scotoma on hazard detection in binocular viewing.

Field Expansion with Multiplexing Prism Glasses Improves Pedestrian Detection for Acquired Monocular Vision.

Purpose: Patients with acquired monocular vision (AMV) lose vision in the temporal crescent on the side of the blind eye. This visual field loss affects patients' ability to detect potential hazards in the blind field. Mounting a base-in multiplexing prism (MxP) on the nasal side of the seeing eye can provide true field expansion and enable detection of potential collision hazards. We evaluated the efficacy of the MxP glasses in a virtual reality walking environment. Methods: A three-dimensional printed clip-on MxP holder that can be adjusted for an individual user's facial parameters was developed. Virtual reality walking scenarios were designed to evaluate the effect of MxP field expansion on the detection of a pedestrian approaching from different initial bearing angles and courses. The pedestrian detection rates and response times of 10 participants with simulated AMV (normally sighted participants with one eye patched) and three patients with AMV were measured. Results: The MxP provided true field expansion of about 25°. Participants performed significantly better with the MxP than without the MxP in the pedestrian detection task on their blind field, while their seeing field performance was not significantly different. Conclusions: The MxP glasses for patients with AMV improved the detection of potential collision hazards in the blind field. Translational Relevance: The MxP with an adjustable clip-on holder may help patients with AMV to decrease the risk of collision with other pedestrians.

Pedestrians Accept Shorter Distances to Light Vehicles Than Dark Ones When Crossing the Street.

Does the brightness of an approaching vehicle affect a pedestrian's crossing decision? Thirty participants indicated their street-crossing intentions when facing approaching light or dark vehicles. The experiment was conducted in a real daylight environment and, additionally, in a corresponding virtual one. A real road with actual cars provides high face validity, while a virtual environment ensures the scenario's precise reproducibility and repeatability for each participant. In both settings, participants judged dark vehicles to be a more imminent threat-either closer or moving faster-when compared with light ones. Secondary results showed that participants accepted a significantly shorter time-to-contact when crossing the street in the virtual setting than on the real road.

Design of 45° periscopic visual field expansion device for peripheral field loss.

Patients with visual field loss have difficulty in mobility due to collision with pedestrians/obstacles from the blind side. In order to retrieve the lost visual field, prisms which deflect the field from the blind to the seeing side, have been widely used. However, the deflection power of current clinical Fresnel prisms is limited to ~30° and only provides a 5° eye scanning range to the blind side. This is not sufficient to avoid collision and results in increasing demands for a device with a higher power. In this paper, we propose a novel design and optimization of a higher power prism-like device (cascaded structure of mirror pairs filled with high refractive index) and verify enhanced expansion of up to 45° in optical ray tracing and photorealistic simulations.