Open Iris - An Open Source Framework for Video-Based Eye-Tracking Research and Development.

Eye-tracking is an essential tool in many fields, yet existing solutions are often limited for customized applications due to cost or lack of flexibility. We present OpenIris, an adaptable and user-friendly open-source framework for video-based eye-tracking. OpenIris is developed in C# with modular design that allows further extension and customization through plugins for different hardware systems, tracking, and calibration pipelines. It can be remotely controlled via a network interface from other devices or programs. Eye movements can be recorded online from camera stream or offline post-processing recorded videos. Example plugins have been developed to track eye motion in 3-D, including torsion. Currently implemented binocular pupil tracking pipelines can achieve frame rates of more than 500Hz. With the OpenIris framework, we aim to fill a gap in the research tools available for high-precision and high-speed eye-tracking, especially in environments that require custom solutions that are not currently well-served by commercial eye-trackers.

Quantifying Induced Nystagmus Using a Smartphone Eye Tracking Application (EyePhone).

BACKGROUND: There are ≈5 million annual dizziness visits to US emergency departments, of which vestibular strokes account for over 250 000. The head impulse, nystagmus, and test of skew eye examination can accurately distinguish vestibular strokes from peripheral dizziness. However, the eye-movement signs are subtle, and lack of familiarity and difficulty with recognition of abnormal eye movements are significant barriers to widespread emergency department use. To break this barrier, we sought to assess the accuracy of EyePhone, our smartphone eye-tracking application, for quantifying nystagmus. METHODS AND RESULTS: We prospectively enrolled healthy volunteers and recorded the velocity of induced nystagmus using a smartphone eye-tracking application (EyePhone) and then compared the results with video oculography (VOG). Following a calibration protocol, the participants viewed optokinetic stimuli with incremental velocities (2-12 degrees/s) in 4 directions. We extracted slow phase velocities from EyePhone data in each direction and compared them with the corresponding slow phase velocities obtained by the VOG. Furthermore, we calculated the area under the receiver operating characteristic curve for nystagmus detection by EyePhone. We enrolled 10 volunteers (90% men) with an average age of 30.2±6 years. EyePhone-recorded slow phase velocities highly correlated with the VOG recordings (r=0.98 for horizontal and r=0.94 for vertical). The calibration significantly increased the slope of linear regression for horizontal and vertical slow phase velocities. Evaluating the EyePhone's performance using VOG data with a 2 degrees/s threshold showed an area under the receiver operating characteristic curve of 0.87 for horizontal and vertical nystagmus detection. CONCLUSIONS: We demonstrated that EyePhone could accurately detect and quantify optokinetic nystagmus, similar to the VOG goggles.

Video ocular counter roll: A bedside test of otolith-ocular function.

Here we review the clinical value of a video-oculography test for clinical evaluation of vestibular otolith function. This test is known as the video ocular counter roll (vOCR) and is based on measurement of torsional vestibulo-ocular reflex with a lateral head tilt. The vOCR test consists of a simple maneuver during which the head and torso are tilted en bloc by the examiner. The pattern of vOCR deficit among patients highlights its clinical value in identifying the stage of vestibular loss and recovery. The quick application of vOCR allows examination of otolith-ocular function and assessment of vestibular recovery at the bedside.

Minimal reporting guideline for research involving eye tracking (2023 edition).

A guideline is proposed that comprises the minimum items to be reported in research studies involving an eye tracker and human or non-human primate participant(s). This guideline was developed over a 3-year period using a consensus-based process via an open invitation to the international eye tracking community. This guideline will be reviewed at maximum intervals of 4 years.

The influence of scene tilt on saccade directions is amplitude dependent.

When exploring a visual scene, humans make more saccades in the horizontal direction than any other direction. While many have shown that the horizontal saccade bias rotates in response to scene tilt, it is unclear whether this effect depends on saccade amplitude. We addressed this question by examining the effect of image tilt on the saccade direction distributions recorded during freely viewing natural scenes. Participants (n = 20) viewed scenes tilted at -30°, 0°, and 30°. Saccade distributions during free viewing rotated by an angle of 12.1° ± 6.7° (t(19) = 8.04, p < 0.001) in the direction of the image tilt. When we partitioned the saccades according to their amplitude we found that small amplitude saccades occurred most in the horizontal direction while large amplitude saccades were more oriented to the scene tilt (p < 0.001). To further study the characteristics of small saccades and how they are affected by scene tilt, we looked at the effect of image tilt on small fixational saccades made while fixating a central target amidst a larger scene and found that fixational saccade distributions did not rotate with scene tilt (-0.3° ±1.7° degrees; t(19) = -0.8, p = 0.39). These results suggest a combined effect of two reference frames in saccade generation: one egocentric reference frame that dominates for small saccades, biases them horizontally, and may be common for different tasks, and another allocentric reference frame that biases larger saccades along the orientation of an image during free viewing.

Video Ocular Counter-Roll (vOCR): Otolith-Ocular Function and Compensatory Effect of the Neck Following Vestibular Loss.

OBJECTIVE: Assessment of recovery following vestibular loss has been limited by the lack of bedside measures in clinical settings. Here, we used the video ocular counter-roll (vOCR) test to study otolith-ocular function and compensatory effect of neck proprioception in patients at different stages of vestibular loss. STUDY DESIGN: Case-control study. SETTING: Tertiary care center. METHODS: Fifty-six subjects were recruited including patients with acute (9 ± 2 days [mean ± standard error of mean]), subacute (61 ± 11 days), and chronic (1009 ± 266 days) unilateral loss of vestibular function, as well as a group of healthy controls. We used a video-oculography method based on tracking the iris for vOCR measurement. To examine the effect of neck inputs, vOCR was recorded during two simple tilt maneuvers in all subjects while seated: 30° head-on-body tilt and 30° head-and-body tilt. RESULTS: The vOCR responses evolved at different stages following vestibular loss with improvement of the gains in the chronic stage. The deficit was more pronounced when the whole body was tilted (acute: 0.08 ± 0.01, subacute: 0.11 ± 0.01, chronic: 0.13 ± 0.02, healthy control: 0.18 ± 0.01), and the gain of vOCR improved when the head was tilted on the body (acute: 0.11 ± 0.01, subacute: 0.14 ± 0.01, chronic: 0.13 ± 0.02, healthy control: 0.17 ± 0.01). The time course of vOCR response was affected as well with reduced amplitude and slower response in the acute stage of vestibular loss. CONCLUSION: The vOCR test can be valuable as a clinical marker to measure vestibular recovery and compensatory effect of neck proprioception in patients at different stages following loss of vestibular function.

Measuring the Influence of Magnetic Vestibular Stimulation on Nystagmus, Self-Motion Perception, and Cognitive Performance in a 7T MRT.

Strong magnetic fields induce dizziness, vertigo, and nystagmus due to Lorentz forces acting on the cupula in the semi-circular canals, an effect called magnetic vestibular stimulation (MVS). In this article, we present an experimental setup in a 7T MRT scanner (MRI scanner) that allows the investigation of the influence of strong magnetic fields on nystagmus as well as perceptual and cognitive responses. The strength of MVS is manipulated by altering the head positions of the participants. The orientation of the participants' semicircular canals with respect to the static magnetic field is assessed by combining a 3D magnetometer and 3D constructive interference in steady-state (3D-CISS) images. This approach allows to account for intra- and inter-individual differences in participants' responses to MVS. In the future, MVS can be useful for clinical research, for example, in the investigation of compensatory processes in vestibular disorders. Furthermore, it could foster insights into the interplay between vestibular information and cognitive processes in terms of spatial cognition and the emergence of self-motion percepts under conflicting sensory information. In fMRI studies, MVS can elicit a possible confounding effect, especially in tasks influenced by vestibular information or in studies comparing vestibular patients with healthy controls.

Deep Learning Model for Static Ocular Torsion Detection Using Synthetically Generated Fundus Images.

Purpose: The objective of the study is to develop deep learning models using synthetic fundus images to assess the direction (intorsion versus extorsion) and amount (physiologic versus pathologic) of static ocular torsion. Static ocular torsion assessment is an important clinical tool for classifying vertical ocular misalignment; however, current methods are time-intensive with steep learning curves for frontline providers. Methods: We used a dataset (n = 276) of right eye fundus images. The disc-foveal angle was calculated using ImageJ to generate synthetic images via image rotation. Using synthetic datasets (n = 12,740 images per model) and transfer learning (the reuse of a pretrained deep learning model on a new task), we developed a binary classifier (intorsion versus extorsion) and a multiclass classifier (physiologic versus pathologic intorsion and extorsion). Model performance was evaluated on unseen synthetic and nonsynthetic data. Results: On the synthetic dataset, the binary classifier had an accuracy and area under the receiver operating characteristic curve (AUROC) of 0.92 and 0.98, respectively, whereas the multiclass classifier had an accuracy and AUROC of 0.77 and 0.94, respectively. The binary classifier generalized well on the nonsynthetic data (accuracy = 0.94; AUROC = 1.00). Conclusions: The direction of static ocular torsion can be detected from synthetic fundus images using deep learning methods, which is key to differentiate between vestibular misalignment (skew deviation) and ocular muscle misalignment (superior oblique palsies). Translational Relevance: Given the robust performance of our models on real fundus images, similar strategies can be adopted for deep learning research in rare neuro-ophthalmologic diseases with limited datasets.

Eye tracking: empirical foundations for a minimal reporting guideline.

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").

Upbeat Nystagmus with an Unusual Velocity-Decreasing and Increasing Waveform: a Sign of Gaze-Holding Dysfunction in the Paramedian Tracts in the Medulla?

We report a patient with spontaneous upbeat nystagmus (UBN) due to an ischemic lesion involving the paramedian tract (PMT) in the medulla. Eye movement recordings, using an infrared video-oculography (VOG) system, showed that the slow phase of the nystagmus was initially velocity-decreasing but gradually became velocity-increasing. Simulation of the nystagmus with a mathematical model supports a role for the PMT in relaying premotor signals for vertical gaze holding to the cerebellum. Our model shows that the disruption in cerebellar input from PMT can lead to the velocity-increasing waveform of the nystagmus, whereas the velocity-decreasing waveform could be related to a mismatch between the innervational commands to the ocular muscles (the pulse and step) needed to hold gaze steady.

Critical Current Density in d-Wave Hubbard Superconductors.

In this work, the Generalized Hubbard Model on a square lattice is applied to evaluate the electrical current density of high critical temperature d-wave superconductors with a set of Hamiltonian parameters allowing them to reach critical temperatures close to 100 K. The appropriate set of Hamiltonian parameters permits us to apply our model to real materials, finding a good quantitative fit with important macroscopic superconducting properties such as the critical superconducting temperature (Tc) and the critical current density (Jc). We propose that much as in a dispersive medium, in which the velocity of electrons can be estimated by the gradient of the dispersion relation ∇ε(k), the electron velocity is proportional to ∇E(k) in the superconducting state (where E(k)=(ε(k)-µ)2+Δ2(k) is the dispersion relation of the quasiparticles, and k is the electron wave vector). This considers the change of ε(k) with respect to the chemical potential (µ) and the formation of pairs that gives rise to an excitation energy gap Δ(k) in the electron density of states across the Fermi level. When ε(k)=µ at the Fermi surface (FS), only the term for the energy gap remains, whose magnitude reflects the strength of the pairing interaction. Under these conditions, we have found that the d-wave symmetry of the pairing interaction leads to a maximum critical current density in the vicinity of the antinodal k-space direction (π,0) of approximately 1.407236×108 A/cm2, with a much greater current density along the nodal direction (π2,π2) of 2.214702×109 A/cm2. These results allow for the establishment of a maximum limit for the critical current density that could be attained by a d-wave superconductor.

Head Orientation Influences Saccade Directions during Free Viewing.

When looking around a visual scene, humans make saccadic eye movements to fixate objects of interest. While the extraocular muscles can execute saccades in any direction, not all saccade directions are equally likely: saccades in horizontal and vertical directions are most prevalent. Here, we asked whether head orientation plays a role in determining saccade direction biases. Study participants (n = 14) viewed natural scenes and abstract fractals (radially symmetric patterns) through a virtual reality headset equipped with eye tracking. Participants' heads were stabilized and tilted at -30°, 0°, or 30° while viewing the images, which could also be tilted by -30°, 0°, and 30° relative to the head. To determine whether the biases in saccade direction changed with head tilt, we calculated polar histograms of saccade directions and cross-correlated pairs of histograms to find the angular displacement resulting in the maximum correlation. During free viewing of fractals, saccade biases largely followed the orientation of the head with an average displacement value of 24° when comparing head upright to head tilt in world-referenced coordinates (t (13) = 17.63, p < 0.001). There was a systematic offset of 2.6° in saccade directions, likely reflecting ocular counter roll (OCR; t (13) = 3.13, p = 0.008). When participants viewed an Earth upright natural scene during head tilt, we found that the orientation of the head still influenced saccade directions (t (13) = 3.7, p = 0.001). These results suggest that nonvisual information about head orientation, such as that acquired by vestibular sensors, likely plays a role in saccade generation.

Modeling the effect of gravity on periodic alternating nystagmus.

Periodic alternating nystagmus (PAN) is a rare oscillatory ocular motor disorder. The effects of gravity on the dynamic behavior of PAN can be studied by monitoring the nystagmus while changing head orientation. Previous studies of patients with PAN reached different conclusions about the effect of changing the orientation of the head relative to gravity on the ongoing PAN, either no effect or a damping of the nystagmus within several minutes. What neuronal circuits could account for the difference in the effects of gravity among PAN patients? We modeled how the brain resolves the tilt-translation ambiguity in normal individuals and added an unstable, oscillatory vestibular system generating PAN. PAN was suppressed in our patient in ear-down positions, in a similar pattern to that of a previously reported patient. This effect was simulated by reducing the gain of the projection of the "rotation feedback" loop to the velocity-storage integrator to approximately 5% of its normal value. With normal "rotation feedback" PAN is expected to dissipate quickly as soon as the head is rotated away from upright position. Moreover, by disconnecting the rotation feedback completely (gain = zero) the model simulated PAN that was reported to be unaffected by gravity. Thus, understanding the effect of this single parameter, the gain of the rotation feedback, can explain the observed variability among our own and previous studies.

aEYE: A deep learning system for video nystagmus detection.

Background: Nystagmus identification and interpretation is challenging for non-experts who lack specific training in neuro-ophthalmology or neuro-otology. This challenge is magnified when the task is performed via telemedicine. Deep learning models have not been heavily studied in video-based eye movement detection. Methods: We developed, trained, and validated a deep-learning system (aEYE) to classify video recordings as normal or bearing at least two consecutive beats of nystagmus. The videos were retrospectively collected from a subset of the monocular (right eye) video-oculography (VOG) recording used in the Acute Video-oculography for Vertigo in Emergency Rooms for Rapid Triage (AVERT) clinical trial (#NCT02483429). Our model was derived from a preliminary dataset representing about 10% of the total AVERT videos (n = 435). The videos were trimmed into 10-sec clips sampled at 60 Hz with a resolution of 240 × 320 pixels. We then created 8 variations of the videos by altering the sampling rates (i.e., 30 Hz and 15 Hz) and image resolution (i.e., 60 × 80 pixels and 15 × 20 pixels). The dataset was labeled as "nystagmus" or "no nystagmus" by one expert provider. We then used a filtered image-based motion classification approach to develop aEYE. The model's performance at detecting nystagmus was calculated by using the area under the receiver-operating characteristic curve (AUROC), sensitivity, specificity, and accuracy. Results: An ensemble between the ResNet-soft voting and the VGG-hard voting models had the best performing metrics. The AUROC, sensitivity, specificity, and accuracy were 0.86, 88.4, 74.2, and 82.7%, respectively. Our validated folds had an average AUROC, sensitivity, specificity, and accuracy of 0.86, 80.3, 80.9, and 80.4%, respectively. Models created from the compressed videos decreased in accuracy as image sampling rate decreased from 60 Hz to 15 Hz. There was only minimal change in the accuracy of nystagmus detection when decreasing image resolution and keeping sampling rate constant. Conclusion: Deep learning is useful in detecting nystagmus in 60 Hz video recordings as well as videos with lower image resolutions and sampling rates, making it a potentially useful tool to aid future automated eye-movement enabled neurologic diagnosis.

Measures of Spatial Orientation: Spatial Bias Analogs in Visual and Haptic Tasks.

The primary sensory modality for probing spatial orientation can vary among psychophysical tasks. In the subjective visual vertical (SVV) task, a visual stimulus is used to measure perceived vertical orientation, while a haptic stimulus is used in the subjective haptic vertical (SHV) task. Here we examined disparity in SHV and SVV task results and asked whether it could be related to biases in probing different spatial estimates by each task. Forty-two healthy volunteers (mean ± SD age, 25 ± 10 years; 19 females; 21 left handed) were recruited. The effect of a task to measure spatial orientation was calculated as the difference between SHV and SVV values, and with the head upright and tilted 20° laterally. There was a task bias regardless of head position related to hand use in the haptic task but not handedness (mean head upright ± SEM: left hand, -3.7 ± 1.1°; right hand, 7.9 ± 1.0°). When this task bias was subtracted out, there was a similar spatial bias using each hand in the SHV task that was also comparable to the SVV task (mean head with left tilt: left hand, 3.9 ± 0.7°; right hand, 4.4 ± 0.7°; SVV, 4.9 ± 0.7°; mean head with right tilt: left hand, -4.6 ± 0.9°; right hand, -4.6 ± 0.8°; SVV, -4.7 ± 1.0°). These findings show that the disparity in visual and haptic measures of spatial orientation is primarily related to a modality-specific bias, and once the effect of hand use is removed from the haptic measurements, the spatial bias becomes comparable to the visual task.

Eye and Head Movement Recordings Using Smartphones for Telemedicine Applications: Measurements of Accuracy and Precision.

Objective: Smartphones have shown promise in the assessment of neuro-ophthalmologic and vestibular disorders. We have shown that the head impulse test results recorded using our application are comparable with measurements from clinical video-oculography (VOG) goggles. The smartphone uses ARKit's capability to acquire eye and head movement positions without the need of performing a calibration as in most eye-tracking devices. Here, we measure the accuracy and precision of the eye and head position recorded using our application. Methods: We enrolled healthy volunteers and asked them to direct their eyes, their heads, or both to targets on a wall at known eccentricities while recording their head and eye movements with our smartphone application. We measured the accuracy as the error between the eye or head movement measurement and the location of each target and the precision as the standard deviation of the eye or head position for each of the target positions. Results: The accuracy of head recordings (15% error) was overall better than the accuracy of eye recordings (23% error). We also found that the accuracy for horizontal eye movements (17% error) was better than for vertical (27% error). Precision was also better for head movement (0.8 degrees) recordings than eye movement recordings (1.3 degrees) and variability tended to increase with eccentricity. Conclusion: Our results provide basic metrics evaluating the utility of smartphone applications in the quantitative assessment of head and eye movements. While the new method may not replace the more accurate dedicated VOG devices, they provide a more accessible quantitative option. It may be advisable to include a calibration recording together with any planned clinical test to improve the accuracy.

Vestibular Test Results in Patients With Horizontal Canal Benign Paroxysmal Positional Vertigo.

Introduction While the mechanism of posterior canal benign paroxysmal positional vertigo (BPPV) is widely accepted as canalolithiasis, the pathophysiology of horizontal canal BPPV remains controversial. We seek to analyze vestibular test results of patients with horizontal canal BPPV with ageotropic nystagmus (AHC) and geotropic nystagmus (GHC) in comparison to patients with posterior canal BPPV (PC) to better understand its pathophysiology. Methods In a retrospective chart review of adults with BPPV at a tertiary referral balance center, we reviewed the clinical characteristics and compared videonystagmography, caloric, rotary chair, subjective visual vertical (SVV)/ subjective visual horizontal (SVH), and vestibular evoked myogenic potential (VEMP) results between groups. Results We included 11 AHC and seven GHC patients and randomly selected 20 PC patients as the comparison group. All groups had a high rate of migraine and low rates of diabetes and head trauma, but no difference between groups. Ipsilateral caloric weakness was more prevalent in the GHC group compared to the PC group (p=0.02). One of two AHC patients and both GHC patients who had SVV/SVH testing had abnormal findings. The only AHC patient who had ocular VEMP testing had abnormal results. Additionally, we observed a significant downbeating component to nystagmus (4 deg/sec or greater) exclusively in the AHC group (5/10 patients, p=0.001). Conclusions Patients with AHC and GHC have unique vestibular testing results. In particular, only AHC patients showed a downbeating component to their nystagmus, which may suggest utricular dysfunction in the pathophysiology of AHC.

Psychophysical Haptic Measurement of Vertical Perception: Elucidating a Hand Sensory Bias.

The primary sensory modality for probing spatial perception can vary among psychophysical paradigms. In the subjective visual vertical (SVV) task, the brain must account for the position of the eye within the orbit to generate an estimate of a visual line orientation, whereas in the subjective haptic vertical (SHV) task, the position of the hand is used to sense the orientation of a haptic bar. Here we investigated whether a hand sensory bias can affect SHV measurement. We measured SHV in 12 subjects (6 left-handed and 6 right-handed) with a forced-choice paradigm using their left and right hands separately. The SHV measurement was less accurate than the SVV measurements (-0.6 ± 0.7) and it was biased in the direction of the hand used in the task but was not affected by handedness; SHV left hand -6.8 ± 2.1° (left-handed -7.9 ± 3.6°, right-handed -5.8 ± 2.5°) and right hand 9.8 ± 1.5° (left-handed 7.4 ± 2.2°, right-handed 12.3 ± 1.8°). SHV measurement with the same hand was also affected by the haptic bar placement on the left or right side versus midline, showing a side effect (left vs midline -2.0 ± 1.3°, right vs midline 3.8 ± 1.7°). Midline SHV measures using the left and right hands were different, confirming a laterality effect (left hand -4.5 ± 1.7°, right hand 6.4 ± 2.0°). These results demonstrate a sensory bias in SHV measurement related to the effects of both hand-in-body (i.e., right vs left hand) and hand-in-space positions. Such modality-specific bias may result in disparity between SHV and SVV measurements, and therefore cannot be generalized to vertical or spatial perception.