Accuracy and Usability of Smartphone-Based Distance Estimation Approaches for Visual Assistive Technology Development.

Goal: Distance information is highly requested in assistive smartphone Apps by people who are blind or low vision (PBLV). However, current techniques have not been evaluated systematically for accuracy and usability. Methods: We tested five smartphone-based distance-estimation approaches in the image center and periphery at 1-3 meters, including machine learning (CoreML), infrared grid distortion (IR_self), light detection and ranging (LiDAR_back), and augmented reality room-tracking on the front (ARKit_self) and back-facing cameras (ARKit_back). Results: For accuracy in the image center, all approaches had <±2.5 cm average error, except CoreML which had ±5.2-6.2 cm average error at 2-3 meters. In the periphery, all approaches were more inaccurate, with CoreML and IR_self having the highest average errors at ±41 cm and ±32 cm respectively. For usability, CoreML fared favorably with the lowest central processing unit usage, second lowest battery usage, highest field-of-view, and no specialized sensor requirements. Conclusions: We provide key information that helps design reliable smartphone-based visual assistive technologies to enhance the functionality of PBLV.

Wearables for persons with blindness and low vision: form factor matters.

Based on statistics from the WHO and the International Agency for the Prevention of Blindness, an estimated 43.3 million people have blindness and 295 million have moderate and severe vision impairment globally as of 2020, statistics expected to increase to 61 million and 474 million respectively by 2050, staggering numbers. Blindness and low vision (BLV) stultify many activities of daily living, as sight is beneficial to most functional tasks. Assistive technologies for persons with blindness and low vision (pBLV) consist of a wide range of aids that work in some way to enhance one's functioning and support independence. Although handheld and head-mounted approaches have been primary foci when building new platforms or devices to support function and mobility, this perspective reviews potential shortcomings of these form factors or embodiments and posits that a body-centered approach may overcome many of these limitations.

Training AI to Recognize Objects of Interest to the Blind and Low Vision Community.

Recent object detection models show promising advances in their architecture and performance, expanding potential applications for the benefit of persons with blindness or low vision (pBLV). However, object detection models are usually trained on generic data rather than datasets that focus on the needs of pBLV. Hence, for applications that locate objects of interest to pBLV, object detection models need to be trained specifically for this purpose. Informed by prior interviews, questionnaires, and Microsoft's ORBIT research, we identified thirty-five objects pertinent to pBLV. We employed this user-centric feedback to gather images of these objects from the Google Open Images V6 dataset. We subsequently trained a YOLOv5x model with this dataset to recognize these objects of interest. We demonstrate that the model can identify objects that previous generic models could not, such as those related to tasks of daily functioning - e.g., coffee mug, knife, fork, and glass. Crucially, we show that careful pruning of a dataset with severe class imbalances leads to a rapid, noticeable improvement in the overall performance of the model by two-fold, as measured using the mean average precision at the intersection over union thresholds from 0.5 to 0.95 (mAP50-95). Specifically, mAP50-95 improved from 0.14 to 0.36 on the seven least prevalent classes in the training dataset. Overall, we show that careful curation of training data can improve training speed and object detection outcomes. We show clear directions on effectively customizing training data to create models that focus on the desires and needs of pBLV.Clinical Relevance- This work demonstrated the benefits of developing assistive AI technology customized to individual users or the wider BLV community.

A Smart Service System for Spatial Intelligence and Onboard Navigation for Individuals with Visual Impairment (VIS4ION Thailand): study protocol of a randomized controlled trial of visually impaired students at the Ratchasuda College, Thailand.

BACKGROUND: Blind/low vision (BLV) severely limits information about our three-dimensional world, leading to poor spatial cognition and impaired navigation. BLV engenders mobility losses, debility, illness, and premature mortality. These mobility losses have been associated with unemployment and severe compromises in quality of life. VI not only eviscerates mobility and safety but also, creates barriers to inclusive higher education. Although true in almost every high-income country, these startling facts are even more severe in low- and middle-income countries, such as Thailand. We aim to use VIS4ION (Visually Impaired Smart Service System for Spatial Intelligence and Onboard Navigation), an advanced wearable technology, to enable real-time access to microservices, providing a potential solution to close this gap and deliver consistent and reliable access to critical spatial information needed for mobility and orientation during navigation. METHODS: We are leveraging 3D reconstruction and semantic segmentation techniques to create a digital twin of the campus that houses Mahidol University's disability college. We will do cross-over randomization, and two groups of randomized VI students will deploy this augmented platform in two phases: a passive phase, during which the wearable will only record location, and an active phase, in which end users receive orientation cueing during location recording. A group will perform the active phase first, then the passive, and the other group will experiment reciprocally. We will assess for acceptability, appropriateness, and feasibility, focusing on experiences with VIS4ION. In addition, we will test another cohort of students for navigational, health, and well-being improvements, comparing weeks 1 to 4. We will also conduct a process evaluation according to the Saunders Framework. Finally, we will extend our computer vision and digital twinning technique to a 12-block spatial grid in Bangkok, providing aid in a more complex environment. DISCUSSION: Although electronic navigation aids seem like an attractive solution, there are several barriers to their use; chief among them is their dependence on either environmental (sensor-based) infrastructure or WiFi/cell "connectivity" infrastructure or both. These barriers limit their widespread adoption, particularly in low-and-middle-income countries. Here we propose a navigation solution that operates independently of both environmental and Wi-Fi/cell infrastructure. We predict the proposed platform supports spatial cognition in BLV populations, augmenting personal freedom and agency, and promoting health and well-being. TRIAL REGISTRATION: ClinicalTrials.gov under the identifier: NCT03174314, Registered 2017.06.02.

UNav: An Infrastructure-Independent Vision-Based Navigation System for People with Blindness and Low Vision.

Vision-based localization approaches now underpin newly emerging navigation pipelines for myriad use cases, from robotics to assistive technologies. Compared to sensor-based solutions, vision-based localization does not require pre-installed sensor infrastructure, which is costly, time-consuming, and/or often infeasible at scale. Herein, we propose a novel vision-based localization pipeline for a specific use case: navigation support for end users with blindness and low vision. Given a query image taken by an end user on a mobile application, the pipeline leverages a visual place recognition (VPR) algorithm to find similar images in a reference image database of the target space. The geolocations of these similar images are utilized in a downstream task that employs a weighted-average method to estimate the end user's location. Another downstream task utilizes the perspective-n-point (PnP) algorithm to estimate the end user's direction by exploiting the 2D-3D point correspondences between the query image and the 3D environment, as extracted from matched images in the database. Additionally, this system implements Dijkstra's algorithm to calculate a shortest path based on a navigable map that includes the trip origin and destination. The topometric map used for localization and navigation is built using a customized graphical user interface that projects a 3D reconstructed sparse map, built from a sequence of images, to the corresponding a priori 2D floor plan. Sequential images used for map construction can be collected in a pre-mapping step or scavenged through public databases/citizen science. The end-to-end system can be installed on any internet-accessible device with a camera that hosts a custom mobile application. For evaluation purposes, mapping and localization were tested in a complex hospital environment. The evaluation results demonstrate that our system can achieve localization with an average error of less than 1 m without knowledge of the camera's intrinsic parameters, such as focal length.

King-Devick Test Performance and Cognitive Dysfunction after Concussion: A Pilot Eye Movement Study.

(1) Background: The King-Devick (KD) rapid number naming test is sensitive for concussion diagnosis, with increased test time from baseline as the outcome measure. Eye tracking during KD performance in concussed individuals shows an association between inter-saccadic interval (ISI) (the time between saccades) prolongation and prolonged testing time. This pilot study retrospectively assesses the relation between ISI prolongation during KD testing and cognitive performance in persistently-symptomatic individuals post-concussion. (2) Results: Fourteen participants (median age 34 years; 6 women) with prior neuropsychological assessment and KD testing with eye tracking were included. KD test times (72.6 ± 20.7 s) and median ISI (379.1 ± 199.1 msec) were prolonged compared to published normative values. Greater ISI prolongation was associated with lower scores for processing speed (WAIS-IV Coding, r = 0.72, p = 0.0017), attention/working memory (Trails Making A, r = -0.65, p = 0.006) (Digit Span Forward, r = 0.57, p = -0.017) (Digit Span Backward, r= -0.55, p = 0.021) (Digit Span Total, r = -0.74, p = 0.001), and executive function (Stroop Color Word Interference, r = -0.8, p = 0.0003). (3) Conclusions: This pilot study provides preliminary evidence suggesting that cognitive dysfunction may be associated with prolonged ISI and KD test times in concussion.

Dysfunctional mode switching between fixation and saccades: collaborative insights into two unusual clinical disorders.

Voluntary rapid eye movements (saccades) redirect the fovea toward objects of visual interest. The saccadic system can be considered as a dual-mode system: in one mode the eye is fixating, in the other it is making a saccade. In this review, we consider two examples of dysfunctional saccades, interrupted saccades in late-onset Tay-Sachs disease and gaze-position dependent opsoclonus after concussion, which fail to properly shift between fixation and saccade modes. Insights and benefits gained from bi-directional collaborative exchange between clinical and basic scientists are emphasized. In the case of interrupted saccades, existing mathematical models were sufficiently detailed to provide support for the cause of interrupted saccades. In the case of gaze-position dependent opsoclonus, existing models could not explain the behavior, but further development provided a reasonable hypothesis for the mechanism underlying the behavior. Collaboration between clinical and basic science is a rich source of progress for developing biologically plausible models and understanding neurological disease. Approaching a clinical problem with a specific hypothesis (model) in mind often prompts new experimental tests and provides insights into basic mechanisms.

How sandbag-able are concussion sideline assessments? A close look at eye movements to uncover strategies.

Background: Sideline diagnostic tests for concussion are vulnerable to volitional poor performance ("sandbagging") on baseline assessments, motivated by desire to subvert concussion detection and potential removal from play. We investigated eye movements during sandbagging versus best effort on the King-Devick (KD) test, a rapid automatized naming (RAN) task.Methods: Participants performed KD testing during oculography following instructions to sandbag or give best effort.Results: Twenty healthy participants without concussion history were included (mean age 27 ± 8 years). Sandbagging resulted in longer test times (89.6 ± 39.2 s vs 48.2 ± 8.5 s, p < .001), longer inter-saccadic intervals (459.5 ± 125.4 ms vs 311.2 ± 79.1 ms, p < .001) and greater numbers of saccades (171.4 ± 47 vs 138 ± 24.2, p < .001) and reverse saccades (wrong direction for reading) (21.2% vs 11.3%, p < .001). Sandbagging was detectable using a logistic model with KD times as the only predictor, though more robustly detectable using eye movement metrics.Conclusions: KD sandbagging results in eye movement differences that are detectable by eye movement recordings and suggest an invalid test score. Objective eye movement recording during the KD test shows promise for distinguishing between best effort and post-injury performance, as well as for identifying sandbagging red flags.

Art therapy for Parkinson's disease.

OBJECTIVE: To explore the potential rehabilitative effect of art therapy and its underlying mechanisms in Parkinson's disease (PD). METHODS: Observational study of eighteen patients with PD, followed in a prospective, open-label, exploratory trial. Before and after twenty sessions of art therapy, PD patients were assessed with the UPDRS, Pegboard Test, Timed Up and Go Test (TUG), Beck Depression Inventory (BDI), Modified Fatigue Impact Scale and PROMIS-Self-Efficacy, Montreal Cognitive Assessment, Rey-Osterrieth Complex Figure Test (RCFT), Benton Visual Recognition Test (BVRT), Navon Test, Visual Search, and Stop Signal Task. Eye movements were recorded during the BVRT. Resting-state functional MRI (rs-fMRI) was also performed to assess functional connectivity (FC) changes within the dorsal attention (DAN), executive control (ECN), fronto-occipital (FOC), salience (SAL), primary and secondary visual (V1, V2) brain networks. We also tested fourteen age-matched healthy controls at baseline. RESULTS: At baseline, PD patients showed abnormal visual-cognitive functions and eye movements. Analyses of rs-fMRI showed increased functional connectivity within DAN and ECN in patients compared to controls. Following art therapy, performance improved on Navon test, eye tracking, and UPDRS scores. Rs-fMRI analysis revealed significantly increased FC levels in brain regions within V1 and V2 networks. INTERPRETATION: Art therapy improves overall visual-cognitive skills and visual exploration strategies as well as general motor function in patients with PD. The changes in brain connectivity highlight a functional reorganization of visual networks.

Looking "Cherry Red Spot Myoclonus" in the Eyes: Clinical Phenotype, Treatment Response, and Eye Movements in Sialidosis Type 1.

Sialidosis type 1 is a rare lysosomal storage disorder caused by mutations of the neuraminidase gene. Specific features suggesting this condition include myoclonus, ataxia and macular cherry-red spots. However, phenotypic variability exists. Here, we present detailed clinical and video description of three patients with this rare condition. We also provide an in-depth characterization of eye movement abnormalities, as an additional tool to investigate pathophysiological mechanisms and to facilitate diagnosis. In our patients, despite phenotypic differences, eye movement deficits largely localized to the cerebellum.

The complexity of eye-hand coordination: a perspective on cortico-cerebellar cooperation.

BACKGROUND: Eye-hand coordination (EHC) is a sophisticated act that requires interconnected processes governing synchronization of ocular and manual motor systems. Precise, timely and skillful movements such as reaching for and grasping small objects depend on the acquisition of high-quality visual information about the environment and simultaneous eye and hand control. Multiple areas in the brainstem and cerebellum, as well as some frontal and parietal structures, have critical roles in the control of eye movements and their coordination with the head. Although both cortex and cerebellum contribute critical elements to normal eye-hand function, differences in these contributions suggest that there may be separable deficits following injury. METHOD: As a preliminary assessment for this perspective, we compared eye and hand-movement control in a patient with cortical stroke relative to a patient with cerebellar stroke. RESULT: We found the onset of eye and hand movements to be temporally decoupled, with significant decoupling variance in the patient with cerebellar stroke. In contrast, the patient with cortical stroke displayed increased hand spatial errors and less significant temporal decoupling variance. Increased decoupling variance in the patient with cerebellar stroke was primarily due to unstable timing of rapid eye movements, saccades. CONCLUSION: These findings highlight a perspective in which facets of eye-hand dyscoordination are dependent on lesion location and may or may not cooperate to varying degrees. Broadly speaking, the results corroborate the general notion that the cerebellum is instrumental to the process of temporal prediction for eye and hand movements, while the cortex is instrumental to the process of spatial prediction, both of which are critical aspects of functional movement control.

Modeling gaze position-dependent opsoclonus.

Opsoclonus/flutter (O/F) is a rare disorder of the saccadic system. Previously, we modeled O/F that developed in a patient following abuse of anabolic steroids. That model, as in all models of the saccadic system, generates commands to make a change in eye position. Recently, we saw a patient who developed a unique form of opsoclonus following a concussion. The patient had postsaccadic ocular flutter in both directions of gaze, and opsoclonus during fixation and pursuit in the left hemifield. A new model of the saccadic system is needed to account for this gaze-position dependent O/F. We started with our prior model, which contains two key elements, mutual inhibition between inhibitory burst neurons on both sides and a prolonged reactivation time of the omnipause neurons (OPNs). We included new inputs to the OPNs from the nucleus prepositus hypoglossi and the frontal eye fields, which contain position-dependent neurons. This provides a mechanism for delaying OPN reactivation, and creating a gaze-position dependence. A simplified pursuit system was also added, the output of which inhibits the OPNs, providing a mechanism for gaze-dependence during pursuit. The rest of the model continues to generate a command to change eye position.

Eye-hand re-coordination: A pilot investigation of gaze and reach biofeedback in chronic stroke.

Within the domain of motor performance, eye-hand coordination centers on close relationships between visuo-perceptual, ocular and appendicular motor systems. This coordination is critically dependent on a cycle of feedforward predictions and feedback-based corrective mechanisms. While intrinsic feedback harnesses naturally available movement-dependent sensory channels to modify movement errors, extrinsic feedback may be provided synthetically by a third party for further supplementation. Extrinsic feedback has been robustly explored in hand-focused, motor control studies, such as through computer-based visual displays, highlighting the spatial errors of reaches. Similar attempts have never been tested for spatial errors related to eye movements, despite the potential to alter ocular motor performance. Stroke creates motor planning deficits, resulting in the inability to generate predictions of motor performance. In this study involving visually guided pointing, we use an interactive computer display to provide extrinsic feedback of hand endpoint errors in an initial baseline experiment (pre-) and then feedback of both eye and hand errors in a second experiment (post-) to chronic stroke participants following each reach trial. We tested the hypothesis that extrinsic feedback of eye and hand would improve predictions and therefore feedforward control. We noted this improvement through gains in the spatial and temporal aspects of eye-hand coordination or an improvement in the decoupling noted as incoordination post-stroke in previous studies, returning performance toward healthy, control behavior. More specifically, results show that stroke participants, following the interventional feedback for eye and hand, improved both their accuracy and timing. This was evident through a temporal re-synchronization between eyes and hands, improving correlations between movement timing, as well as reducing the overall time interval (delay) between effectors. These experiments provide a strong indication that an extrinsic feedback intervention at appropriate therapeutic doses may improve eye-hand coordination during stroke rehabilitation.

Eye position-dependent opsoclonus in mild traumatic brain injury.

Opsoclonus consists of bursts of involuntary, multidirectional, back-to-back saccades without an intersaccadic interval. We report a 60-year-old man with post-concussive headaches and disequilibrium who had small amplitude opsoclonus in left gaze, along with larger amplitude flutter during convergence. Examination was otherwise normal and brain MRI was unremarkable. Video-oculography demonstrated opsoclonus predominantly in left gaze and during pursuit in the left hemifield, which improved as post-concussive symptoms improved. Existing theories of opsoclonus mechanisms do not account for this eye position-dependence. We discuss theoretical mechanisms of this behavior, including possible dysfunction of frontal eye field and/or cerebellar vermis neurons; review ocular oscillations in traumatic brain injury; and consider the potential relationship between the larger amplitude flutter upon convergence and post-traumatic ocular oscillations.

Efficiently Recording the Eye-Hand Coordination to Incoordination Spectrum.

The objective analysis of eye movements has a significant history and has been long proven to be an important research tool in the setting of brain injury. Quantitative recordings have a strong capacity to screen diagnostically. Concurrent examinations of the eye and upper limb movements directed toward shared functional goals (e.g., eye-hand coordination) serve as an additional robust biomarker-laden path to capture and interrogate neural injury, including acquired brain injury (ABI). While quantitative dual-effector recordings in 3-D afford ample opportunities within ocular-manual motor investigations in the setting of ABI, the feasibility of such dual recordings for both eye and hand is challenging in pathological settings, particularly when approached with research-grade rigor. Here we describe the integration of an eye tracking system with a motion tracking system intended primarily for limb control research to study a natural behavior. The protocol enables the investigation of unrestricted, three-dimensional (3D) eye-hand coordination tasks. More specifically, we review a method to assess eye-hand coordination in visually guided saccade-to-reach tasks in subjects with chronic middle cerebral artery (MCA) stroke and compare them to healthy controls. Special attention is paid to the specific eye- and limb-tracking system properties in order to obtain high fidelity data from participants post-injury. Sampling rate, accuracy, permissible head movement range given anticipated tolerance and the feasibility of use were several of the critical properties considered when selecting an eye tracker and an approach. The limb tracker was selected based on a similar rubric but included the need for 3-D recording, dynamic interaction and a miniaturized physical footprint. The quantitative data provided by this method and the overall approach when executed correctly has tremendous potential to further refine our mechanistic understanding of eye-hand control and help inform feasible diagnostic and pragmatic interventions within the neurological and rehabilitative practice.

Did I do that? Detecting a perturbation to visual feedback in a reaching task.

The motor system executes actions in a highly stereotyped manner despite the high number of degrees of freedom available. Studies of motor adaptation leverage this fact by disrupting, or perturbing, visual feedback to measure how the motor system compensates. To elicit detectable effects, perturbations are often large compared to trial-to-trial reach endpoint variability. However, awareness of large perturbations can elicit qualitatively different compensation processes than unnoticeable ones can. The current experiment measures the perturbation detection threshold, and investigates how humans combine proprioception and vision to decide whether displayed reach endpoint errors are self-generated only, or are due to experimenter-imposed perturbation. We scaled or rotated the position of the visual feedback of center-out reaches to targets and asked subjects to indicate whether visual feedback was perturbed. Subjects detected perturbations when they were at least 1.5 times the standard deviation of trial-to-trial endpoint variability. In contrast to previous studies, subjects suboptimally combined vision and proprioception. Instead of using proprioceptive input, they responded based on the final (possibly perturbed) visual feedback. These results inform methodology in motor system experimentation, and more broadly highlight the ability to attribute errors to one's own motor output and combine visual and proprioceptive feedback to make decisions.

The effect of linguistic background on rapid number naming: implications for native versus non-native English speakers on sideline-focused concussion assessments.

OBJECTIVE: To determine if native English speakers (NES) perform differently compared to non-native English speakers (NNES) on a sideline-focused rapid number naming task. A secondary aim was to characterize objective differences in eye movement behaviour between cohorts. BACKGROUND: The King-Devick (KD) test is a rapid number-naming task in which numbers are read from left-to-right. This performance measure adds vision-based assessment to sideline concussion testing. Reading strategies differ by language. Concussion may also impact language and attention. Both factors may affect test performance. METHODS: Twenty-seven healthy  NNES and healthy NES performed a computerized KD test under high-resolution video-oculography.  NNES also performed a Bilingual Dominance Scale (BDS) questionnaire to weight linguistic preferences (i.e., reliance on non-English language(s)). RESULTS: Inter-saccadic intervals were significantly longer in  NNES (346.3 ± 78.3 ms vs. 286.1 ± 49.7 ms, p = 0.001), as were KD test times (54.4 ± 15.1 s vs. 43.8 ± 8.6 s, p = 0.002). Higher BDS scores, reflecting higher native language dominance, were associated with longer inter-saccadic intervals in  NNES. CONCLUSION: These findings have direct implications for the assessment of athlete performance on vision-based and other verbal sideline concussion tests; these results are particularly important given the international scope of sport. Pre-season baseline scores are essential to evaluation in the event of concussion, and performance of sideline tests in the athlete's native language should be considered to optimize both baseline and post-injury test accuracy.