Functional differences between the extraordinary eyes of deep-sea hyperiid amphipods.

The ocean's midwater is a uniquely challenging yet predictable and simple visual environment. The need to see without being seen in this dim, open habitat has led to extraordinary visual adaptations. To understand these adaptations, we compared the morphological and functional differences between the eyes of three hyperiid amphipods-Hyperia galba, Streetsia challengeri and Phronima sedentaria. Combining micro-CT data with computational modelling, we mapped visual field topography and predicted detection distances for visual targets viewed in different directions through mesopelagic depths. Hyperia's eyes provide a wide visual field optimized for spatial vision over short distances, while Phronima's and Streetsia's eyes have the potential to achieve greater sensitivity and longer detection distances using spatial summation. These improvements come at the cost of smaller visual fields, but this loss is compensated for by a second pair of eyes in Phronima and by behaviour in Streetsia. The need to improve sensitivity while minimizing visible eye size to maintain crypsis has likely driven the evolution of hyperiid eye diversity. Our results provide an integrative look at how these elusive animals have adapted to the unique visual challenges of the mesopelagic.

Contrast sensitivity, visual acuity and the effect of behavioural state on optokinetic gain in fiddler crabs.

Most animals rely on visual information for a variety of everyday tasks. The information available to a visual system depends in part on its spatial resolving power and contrast sensitivity. Because of their competing demands for physical space within an eye, these traits cannot simultaneously be improved without increasing overall eye size. The contrast sensitivity function is an integrated measure of visual performance that measures both resolution and contrast sensitivity. Its measurement helps us identify how different species have made a trade-off between contrast sensitivity and spatial resolution. It further allows us to identify the evolutionary drivers of sensory processing and visually mediated behaviour. Here, we measured the contrast sensitivity function of the fiddler crab Gelasimus dampieri using its optokinetic responses to wide-field moving sinusoidal intensity gratings of different orientations, spatial frequencies, contrasts and speeds. We further tested whether the behavioural state of the crabs (i.e. whether crabs are actively walking or not) affects their optokinetic gain and contrast sensitivity. Our results from a group of five crabs suggest a minimum perceived contrast of 6% and a horizontal and vertical visual acuity of 0.4 cyc deg-1 and 0.28 cyc deg-1, respectively, in the crabs' region of maximum optomotor sensitivity. Optokinetic gain increased in moving crabs compared with restrained crabs, adding another example of the importance of naturalistic approaches when studying the performance of animals.

A new computational model illuminates the extraordinary eyes of Phronima.

Vision in the midwater of the open ocean requires animals to perform visual tasks quite unlike those of any other environment. These tasks consist of detecting small, low contrast objects and point sources against a relatively dim and uniform background. Deep-sea animals have evolved many extraordinary visual adaptations to perform these tasks. Linking eye anatomy to specific selective pressures, however, is challenging, not least because of the many difficulties of studying deep-sea animals. Computational modelling of vision, based on detailed morphological reconstructions of animal eyes, along with underwater optics, offers a chance to understand the specific visual capabilities of individual visual systems. Prior to the work presented here, comprehensive models for apposition compound eyes in the mesopelagic, the dominant eye form of crustaceans, were lacking. We adapted a model developed for single-lens eyes and used it to examine how different parameters affect the model's ability to detect point sources and extended objects. This new model also allowed us to examine spatial summation as a means to improve visual performance. Our results identify a trade-off between increased depth range over which eyes function effectively and increased distance at which extended objects can be detected. This trade-off is driven by the size of the ommatidial acceptance angle. We also show that if neighbouring ommatidia have overlapping receptive fields, spatial summation helps with all detection tasks, including the detection of bioluminescent point sources. By applying our model to the apposition compound eyes of Phronima, a mesopelagic hyperiid amphipod, we show that the specialisations of the large medial eyes of Phronima improve both the detection of point sources and of extended objects. The medial eyes outperformed the lateral eyes at every modelled detection task. We suggest that the small visual field size of Phronima's medial eyes and the strong asymmetry between the medial and lateral eyes reflect Phronima's need for effective vision across a large depth range and its habit of living inside a barrel. The barrel's narrow aperture limits the usefulness of a large visual field and has allowed a strong asymmetry between the medial and lateral eyes. The model provides a useful tool for future investigations into the visual abilities of apposition compound eyes in the deep sea.

Fiddler crabs are unique in timing their escape responses based on speed-dependent visual cues.

Predation risk imposes strong selection pressures on visual systems to quickly and accurately identify the position and movement of potential predators.1,2 Many invertebrates and other small animals, however, have limited capacity for distance perception due to their low spatial resolution and closely situated eyes.3,4 Consequently, they often rely on simplified decision criteria, essentially heuristics or "rules of thumb", to make decisions. The visual cues animals use to make escape decisions are surprisingly consistent, especially among arthropods, with the timing of escape commonly triggered by size-dependent visual cues such as angular size or angular size increment.5,6,7,8,9,10 Angular size, however, confuses predator size and distance and provides no information about the speed of the attack. Here, we show that fiddler crabs (Gelasimus dampieri) are unique among the arthropods studied to date as they timed their escape response based on the speed of an object's angular expansion. The crabs responded reliably by running away from visual stimuli that expanded at approximately 1.7 degrees/s, irrespective of stimulus size, speed, or its initial distance from the crabs. Though the threshold expansion speed was consistent across different stimulus conditions, we found that the escape timing was modulated by the elevation at which the stimulus approached, suggesting that other risk factors can bias the expansion speed threshold. The results suggest that the visual escape cues used by arthropods are less conserved than previously thought and that lifestyle and environment are significant drivers determining the escape cues used by different species.

Behavioural and neural responses of crabs show evidence for selective attention in predator avoidance.

Selective attention, the ability to focus on a specific stimulus and suppress distractions, plays a fundamental role for animals in many contexts, such as mating, feeding, and predation. Within natural environments, animals are often confronted with multiple stimuli of potential importance. Such a situation significantly complicates the decision-making process and imposes conflicting information on neural systems. In the context of predation, selectively attending to one of multiple threats is one possible solution. However, how animals make such escape decisions is rarely studied. A previous field study on the fiddler crab, Gelasimus dampieri, provided evidence of selective attention in the context of escape decisions. To identify the underlying mechanisms that guide their escape decisions, we measured the crabs' behavioural and neural responses to either a single, or two simultaneously approaching looming stimuli. The two stimuli were either identical or differed in contrast to represent different levels of threat certainty. Although our behavioural data provides some evidence that crabs perceive signals from both stimuli, we show that both the crabs and their looming-sensitive neurons almost exclusively respond to only one of two simultaneous threats. The crabs' body orientation played an important role in their decision about which stimulus to run away from. When faced with two stimuli of differing contrasts, both neurons and crabs were much more likely to respond to the stimulus with the higher contrast. Our data provides evidence that the crabs' looming-sensitive neurons play an important part in the mechanism that drives their selective attention in the context of predation. Our results support previous suggestions that the crabs' escape direction is calculated downstream of their looming-sensitive neurons by means of a population vector of the looming sensitive neuronal ensemble.

Evidence of predictive selective attention in fiddler crabs during escape in the natural environment.

Selective attention is of fundamental relevance to animals for performing a diversity of tasks such as mating, feeding, predation and avoiding predators. Within natural environments, prey animals are often exposed to multiple, simultaneous threats, which significantly complicates the decision-making process. However, selective attention is rarely studied in complex, natural environments or in the context of escape responses. We therefore asked how relatively simple animals integrate the information from multiple, concurrent threatening events. Do they identify and respond only to what they perceive as the most dangerous threat, or do they respond to multiple stimuli at the same time? Do simultaneous threats evoke an earlier or stronger response than single threats? We investigated these questions by conducting field experiments and compared escape responses of the fiddler crab Gelasimus dampieri when faced with either a single or two simultaneously approaching dummy predators. We used the dummies' approach trajectories to manipulate the threat level; a directly approaching dummy indicated higher risk while a tangentially approaching dummy that passed the crabs at a distance represented a lower risk. The crabs responded later, but on average more often, when approached more directly. However, when confronted with the two dummies simultaneously, the crabs responded as if approached only by the directly approaching dummy. This suggests that the crabs are able to predict how close the dummy's trajectory is to a collision course and selectively suppress their normally earlier response to the less dangerous dummy. We thus provide evidence of predictive selective attention within a natural environment.

Photoreceptors and diurnal variation in spectral sensitivity in the fiddler crab Gelasimus dampieri.

Colour signals, and the ability to detect them, are important for many animals and can be vital to their survival and fitness. Fiddler crabs use colour information to detect and recognise conspecifics, but their colour vision capabilities remain unclear. Many studies have attempted to measure their spectral sensitivity and identify contributing retinular cells, but the existing evidence is inconclusive. We used electroretinogram (ERG) measurements and intracellular recordings from retinular cells to estimate the spectral sensitivity of Gelasimus dampieri and to track diurnal changes in spectral sensitivity. G. dampieri has a broad spectral sensitivity and is most sensitive to wavelengths between 420 and 460 nm. Selective adaptation experiments uncovered an ultraviolet (UV) retinular cell with a peak sensitivity shorter than 360 nm. The species' spectral sensitivity above 400 nm is too broad to be fitted by a single visual pigment and using optical modelling, we provide evidence that at least two medium-wavelength sensitive (MWS) visual pigments are contained within a second blue-green sensitive retinular cell. We also found a ∼25 nm diurnal shift in spectral sensitivity towards longer wavelengths in the evening in both ERG and intracellular recordings. Whether the shift is caused by screening pigment migration or changes in opsin expression remains unclear, but the observation shows the diel dynamism of colour vision in this species. Together, these findings support the notion that G. dampieri possesses the minimum requirement for colour vision, with UV and blue/green receptors, and help to explain some of the inconsistent results of previous research.

A new method for mapping spatial resolution in compound eyes suggests two visual streaks in fiddler crabs.

Visual systems play a vital role in guiding the behaviour of animals. Understanding the visual information animals are able to acquire is therefore key to understanding their visually mediated decision making. Compound eyes, the dominant eye type in arthropods, are inherently low-resolution structures. Their ability to resolve spatial detail depends on sampling resolution (interommatidial angle) and the quality of ommatidial optics. Current techniques for estimating interommatidial angles are difficult, and generally require in vivo measurements. Here, we present a new method for estimating interommatidial angles based on the detailed analysis of 3D micro-computed tomography images of fixed samples. Using custom-made MATLAB software, we determined the optical axes of individual ommatidia and projected these axes into the 3D space around the animal. The combined viewing directions of all ommatidia, estimated from geometrical optics, allowed us to estimate interommatidial angles and map the animal's sampling resolution across its entire visual field. The resulting topographic representations of visual acuity match very closely the previously published data obtained from both fiddler and grapsid crabs. However, the new method provides additional detail that was not previously detectable and reveals that fiddler crabs, rather than having a single horizontal visual streak as is common in flat-world inhabitants, probably have two parallel streaks located just above and below the visual horizon. A key advantage of our approach is that it can be used on appropriately preserved specimens, allowing the technique to be applied to animals such as deep-sea crustaceans that are inaccessible or unsuitable for in vivo approaches.

An autonomous robot inspired by insect neurophysiology pursues moving features in natural environments.

OBJECTIVE: Many computer vision and robotic applications require the implementation of robust and efficient target-tracking algorithms on a moving platform. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. Lightweight and low-powered flying insects, such as dragonflies, track prey or conspecifics within cluttered natural environments, illustrating an efficient biological solution to the target-tracking problem. APPROACH: We used our recent recordings from 'small target motion detector' neurons in the dragonfly brain to inspire the development of a closed-loop target detection and tracking algorithm. This model exploits facilitation, a slow build-up of response to targets which move along long, continuous trajectories, as seen in our electrophysiological data. To test performance in real-world conditions, we implemented this model on a robotic platform that uses active pursuit strategies based on insect behaviour. MAIN RESULTS: Our robot performs robustly in closed-loop pursuit of targets, despite a range of challenging conditions used in our experiments; low contrast targets, heavily cluttered environments and the presence of distracters. We show that the facilitation stage boosts responses to targets moving along continuous trajectories, improving contrast sensitivity and detection of small moving targets against textured backgrounds. Moreover, the temporal properties of facilitation play a useful role in handling vibration of the robotic platform. We also show that the adoption of feed-forward models which predict the sensory consequences of self-movement can significantly improve target detection during saccadic movements. SIGNIFICANCE: Our results provide insight into the neuronal mechanisms that underlie biological target detection and selection (from a moving platform), as well as highlight the effectiveness of our bio-inspired algorithm in an artificial visual system.

Performance of an insect-inspired target tracker in natural conditions.

Robust and efficient target-tracking algorithms embedded on moving platforms, are a requirement for many computer vision and robotic applications. However, deployment of a real-time system is challenging, even with the computational power of modern hardware. As inspiration, we look to biological lightweight solutions-lightweight and low-powered flying insects. For example, dragonflies pursue prey and mates within cluttered, natural environments, deftly selecting their target amidst swarms. In our laboratory, we study the physiology and morphology of dragonfly 'small target motion detector' neurons likely to underlie this pursuit behaviour. Here we describe our insect-inspired tracking model derived from these data and compare its efficacy and efficiency with state-of-the-art engineering models. For model inputs, we use both publicly available video sequences, as well as our own task-specific dataset (small targets embedded within natural scenes). In the context of the tracking problem, we describe differences in object statistics within the video sequences. For the general dataset, our model often locks on to small components of larger objects, tracking these moving features. When input imagery includes small moving targets, for which our highly nonlinear filtering is matched, the robustness outperforms state-of-the-art trackers. In all scenarios, our insect-inspired tracker runs at least twice the speed of the comparison algorithms.

Properties of neuronal facilitation that improve target tracking in natural pursuit simulations.

Although flying insects have limited visual acuity (approx. 1°) and relatively small brains, many species pursue tiny targets against cluttered backgrounds with high success. Our previous computational model, inspired by electrophysiological recordings from insect 'small target motion detector' (STMD) neurons, did not account for several key properties described from the biological system. These include the recent observations of response 'facilitation' (a slow build-up of response to targets that move on long, continuous trajectories) and 'selective attention', a competitive mechanism that selects one target from alternatives. Here, we present an elaborated STMD-inspired model, implemented in a closed loop target-tracking system that uses an active saccadic gaze fixation strategy inspired by insect pursuit. We test this system against heavily cluttered natural scenes. Inclusion of facilitation not only substantially improves success for even short-duration pursuits, but it also enhances the ability to 'attend' to one target in the presence of distracters. Our model predicts optimal facilitation parameters that are static in space and dynamic in time, changing with respect to the amount of background clutter and the intended purpose of the pursuit. Our results provide insights into insect neurophysiology and show the potential of this algorithm for implementation in artificial visual systems and robotic applications.