Passive Polarized Vision for Autonomous Vehicles: A Review.

This review article aims to address common research questions in passive polarized vision for robotics. What kind of polarization sensing can we embed into robots? Can we find our geolocation and true north heading by detecting light scattering from the sky as animals do? How should polarization images be related to the physical properties of reflecting surfaces in the context of scene understanding? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying future directions in passive polarized vision for robotics. After an introduction, three key interconnected areas will be covered in the following sections: embedded polarization imaging; polarized vision for robotics navigation; and polarized vision for scene understanding. We will then discuss how polarized vision, a type of vision commonly used in the animal kingdom, should be implemented in robotics; this type of vision has not yet been exploited in robotics service. Passive polarized vision could be a supplemental perceptive modality of localization techniques to complement and reinforce more conventional ones.

Shaping the energy curves of a servomotor-based hexapod robot.

The advantageous versatility of hexapod robots is often accompanied by high power consumption, while animals have evolved an energy efficient locomotion. However, there are a lack of methods able to compare and apply animals' energetic optimizations to robots. In this study, we applied our method to a full servomotor-based hexapod robot to evaluate its energetic performance. Using an existing framework based on the laws of thermodynamics, we estimated four metrics using a dedicated test bench and a simulated robotic leg. We analyzed the characteristics of a single leg to shape the energetic profile of the full robot to a given task. Energy saving is improved by 10% through continuous duty factor adjustment with a 192% increase in power maximization. Moreover, adjusting the robot's velocity by the step length and associating this with gait switching, reduces the power loss by a further 10% at low-speed locomotion. However, unlike in animals, only one unique optimal operating point has been revealed, which is a disadvantage caused by the low energetic efficiency of servomotor-based hexapods. Thus, these legged robots are severely limited in their capacity to optimally adjust their locomotion to various tasks-a counter-intuitive conclusion for a supposedly versatile robot.

SkyPole-A method for locating the north celestial pole from skylight polarization patterns.

True north can be determined on Earth by three means: magnetic compasses, stars, and via the global navigation satellite systems (GNSS), each of which has its own drawbacks. GNSS are sensitive to jamming and spoofing, magnetic compasses are vulnerable to magnetic interferences, and the stars can be used only at night with a clear sky. As an alternative to these methods, nature-inspired navigational cues are of particular interest. Celestial polarization, which is used by insects such as Cataglyphis ants, can provide useful directional cues. Migrating birds calibrate their magnetic compasses by observing the celestial rotation at night. By combining these cues, we have developed a bioinspired optical method for finding the celestial pole during the daytime. This method, which we have named SkyPole, is based on the rotation of the skylight polarization pattern. A polarimetric camera was used to measure the degree of skylight polarization rotating with the Sun. Image difference processes were then applied to the time-varying measurements in order to determine the north celestial pole's position and thus the observer's latitude and bearing with respect to the true north.

Sensory fusion in the hoverfly righting reflex.

We study how falling hoverflies use sensory cues to trigger appropriate roll righting behavior. Before being released in a free fall, flies were placed upside-down with their legs contacting the substrate. The prior leg proprioceptive information about their initial orientation sufficed for the flies to right themselves properly. However, flies also use visual and antennal cues to recover faster and disambiguate sensory conflicts. Surprisingly, in one of the experimental conditions tested, hoverflies flew upside-down while still actively flapping their wings. In all the other conditions, flies were able to right themselves using two roll dynamics: fast ([Formula: see text]50ms) and slow ([Formula: see text]110ms) in the presence of consistent and conflicting cues, respectively. These findings suggest that a nonlinear sensory integration of the three types of sensory cues occurred. A ring attractor model was developed and discussed to account for this cue integration process.

Quaternion to Euler angles conversion: A direct, general and computationally efficient method.

Current methods of the conversion between a rotation quaternion and Euler angles are either a complicated set of multiple sequence-specific implementations, or a complicated method relying on multiple matrix multiplications. In this paper a general formula is presented for extracting the Euler angles in any desired sequence from a unit quaternion. This is a direct method, in that no intermediate conversion step is required (no quaternion-to-rotation matrix conversion, for example) and it is general because it works with all 12 possible sequences of rotations. A closed formula was first developed for extracting angles in any of the 12 possible sequences, both "Proper Euler angles" and "Tait-Bryan angles". The resulting algorithm was compared with a popular implementation of the matrix-to-Euler angle algorithm, which involves a quaternion-to-matrix conversion in the first computational step. Lastly, a single-page pseudo-code implementation of this algorithm is presented, illustrating its conciseness and straightforward implementation. With an execution speed 30 times faster than the classical method, our algorithm can be of great interest in every aspect.

Insect-Inspired Robots: Bridging Biological and Artificial Systems.

This review article aims to address common research questions in hexapod robotics. How can we build intelligent autonomous hexapod robots that can exploit their biomechanics, morphology, and computational systems, to achieve autonomy, adaptability, and energy efficiency comparable to small living creatures, such as insects? Are insects good models for building such intelligent hexapod robots because they are the only animals with six legs? This review article is divided into three main sections to address these questions, as well as to assist roboticists in identifying relevant and future directions in the field of hexapod robotics over the next decade. After an introduction in section (1), the sections will respectively cover the following three key areas: (2) biomechanics focused on the design of smart legs; (3) locomotion control; and (4) high-level cognition control. These interconnected and interdependent areas are all crucial to improving the level of performance of hexapod robotics in terms of energy efficiency, terrain adaptability, autonomy, and operational range. We will also discuss how the next generation of bioroboticists will be able to transfer knowledge from biology to robotics and vice versa.

Two pursuit strategies for a single sensorimotor control task in blowfly.

Effective visuomotor coordination is a necessary requirement for the survival of many terrestrial, aquatic, and aerial animal species. We studied the kinematics of aerial pursuit in the blowfly Lucilia sericata using an actuated dummy as target for freely flying males. We found that the flies perform target tracking in the horizontal plane and target interception in the vertical plane. Our behavioural data suggest that the flies' trajectory changes are a controlled combination of target heading angle and of the rate of change of the bearing angle. We implemented control laws in kinematic models and found that the contributions of proportional navigation strategy are negligible. We concluded that the difference between horizontal and vertical control relates to the difference in target heading angle the fly keeps constant: 0° in azimuth and 23° in elevation. Our work suggests that male Lucilia control both horizontal and vertical steerings by employing proportional controllers to the error angles. In horizontal plane, this controller operates at time delays as small as 10 ms, the fastest steering response observed in any flying animal, so far.

How do hoverflies use their righting reflex?

When taking off from a sloping surface, flies have to reorient themselves dorsoventrally and stabilize their body by actively controlling their flapping wings. We have observed that righting is achieved solely by performing a rolling manoeuvre. How flies manage to do this has not yet been elucidated. It was observed here for the first time that hoverfly reorientation is entirely achieved within 6 wingbeats (48.8 ms) at angular roll velocities of up to 10×103 deg s-1 and that the onset of their head rotation consistently follows that of their body rotation after a time lag of 16 ms. The insects' body roll was found to be triggered by the asymmetric wing stroke amplitude, as expected. The righting process starts immediately with the first wingbeat and seems unlikely to depend on visual feedback. A dynamic model for the fly's righting reflex is presented, which accounts for the head/body movements and the time lag recorded in these experiments. This model consists of a closed-loop control of the body roll, combined with a feedforward control of the head/body angle. During the righting manoeuvre, a strong coupling seems to exist between the activation of the halteres (which measure the body's angular speed) and the gaze stabilization reflex. These findings again confirm the fundamental role played by the halteres in both body and head stabilization processes.

Polarized skylight-based heading measurements: a bio-inspired approach.

Many insects such as desert ants, crickets, locusts, dung beetles, bees and monarch butterflies have been found to extract their navigation cues from the regular pattern of the linearly polarized skylight. These species are equipped with ommatidia in the dorsal rim area of their compound eyes, which are sensitive to the angle of polarization of the skylight. In the polarization-based robotic vision, most of the sensors used so far comprise high-definition CCD or CMOS cameras topped with linear polarizers. Here, we present a 2-pixel polarization-sensitive visual sensor, which was strongly inspired by the dorsal rim area of desert ants' compound eyes, designed to determine the direction of polarization of the skylight. The spectral sensitivity of this minimalistic sensor, which requires no lenses, is in the ultraviolet range. Five different methods of computing the direction of polarization were implemented and tested here. Our own methods, the extended and AntBot method, outperformed the other three, giving a mean angular error of only 0.62° ± 0.40° (median: 0.24°) and 0.69° ± 0.52° (median: 0.39°), respectively (mean ± standard deviation). The results obtained in outdoor field studies show that our celestial compass gives excellent results at a very low computational cost, which makes it highly suitable for autonomous outdoor navigation purposes.

A novel setup for 3D chasing behavior analysis in free flying flies.

BACKGROUND: Insects catching prey or mates on the wing perform one of the fastest behaviours observed in nature. Some dipteran flies are aerial acrobats specialized to detect, chase and capture their targets within the blink of an eye. Studies of aerial pursuits and its underlying sensorimotor control have been a long-standing subject of interest in neuroethology research. NEW METHOD: We designed an actuated dummy target to trigger chasing flights in male blowflies. Our setup generates arbitrary 2D target trajectories in the horizontal plane combining translation up to 1 m/s and angular rotation up to 720°/s. RESULTS: Using stereovision methods we reconstructed target and pursuer positions every 5 ms with a maximum 3D error of 5 mm. The pursuer's body pitch and yaw angles were resolved within an error range of 6deg. An embedded observation point provides a close-up view of the pursuer's final approach and enables us to measure its body roll angle. We observed banked turns and sideslip which have not been reported for chasing blowflies in the past. COMPARISON WITH EXISTING METHOD(S): Previous studies focused on pursuit along circular paths or interception of translating targets while our method allows us to generate more complex target trajectories. Measurements of body orientation in earlier accounts were limited to the heading direction while we extended the analysis to include the full body orientation during pursuit. CONCLUSIONS: Our setup offers an opportunity to investigate kinematics and governing visual parameters of chasing behaviour in species up to the size of blowflies under a large variety of experimental conditions.

A bio-inspired sighted robot chases like a hoverfly.

Here we present a novel bio-inspired visual processing system, which enables a robot to locate and follow a target, using an artificial compound eye called CurvACE. This visual sensor actively scanned the environment at an imposed frequency (50 Hz) with an angular scanning amplitude of [Formula: see text] and succeeded in locating a textured cylindrical target with hyperacuity, i.e. much finer resolution than the coarse inter-receptor angle of the compound eye. Equipped with this small, lightweight visual scanning sensor, a Mecanum-wheeled mobile robot named ACEbot was able to follow a target at a constant distance by localizing the right and left edges of the target. The localization of the target's contrasted edges is based on a bio-inspired summation of Gaussian receptive fields in the visual system. By means of its auto-adaptive pixels, ACEbot consistently achieved similar pursuit performances under various lighting conditions with a high level of repeatability. The robotic pursuit pattern mimics finely the behavior of the male fly Syritta Pipens L. while pursuing the female. The high similarity in the trajectories as well as the biomimicry of the visual system provides strong support for the hypothesis that flies do maintain center the target and constant its subtended angle during smooth pursuit. Moreover, we discuss the fact that such simple strategy can also provide a trajectory compatible with motion camouflage.

AntBot: A six-legged walking robot able to home like desert ants in outdoor environments.

Autonomous outdoor navigation requires reliable multisensory fusion strategies. Desert ants travel widely every day, showing unrivaled navigation performance using only a few thousand neurons. In the desert, pheromones are instantly destroyed by the extreme heat. To navigate safely in this hostile environment, desert ants assess their heading from the polarized pattern of skylight and judge the distance traveled based on both a stride-counting method and the optic flow, i.e., the rate at which the ground moves across the eye. This process is called path integration (PI). Although many methods of endowing mobile robots with outdoor localization have been developed recently, most of them are still prone to considerable drift and uncertainty. We tested several ant-inspired solutions to outdoor homing navigation problems on a legged robot using two optical sensors equipped with just 14 pixels, two of which were dedicated to an insect-inspired compass sensitive to ultraviolet light. When combined with two rotating polarized filters, this compass was equivalent to two costly arrays composed of 374 photosensors, each of which was tuned to a specific polarization angle. The other 12 pixels were dedicated to optic flow measurements. Results show that our ant-inspired methods of navigation give precise performances. The mean homing error recorded during the overall trajectory was as small as 0.67% under lighting conditions similar to those encountered by ants. These findings show that ant-inspired PI strategies can be used to complement classical techniques with a high level of robustness and efficiency.

Insect-inspired vision for autonomous vehicles.

Flying insects are being studied these days as if they were agile micro air vehicles fitted with smart sensors, requiring very few brain resources. The findings obtained on these natural fliers have proved to be extremely valuable when it comes to designing compact low-weight artificial optical sensors capable of performing visual processing tasks robustly under various environmental conditions (light, clouds, contrast). Here, we review some outstanding bio-inspired visual sensors, which can be used for either detecting motion in the visible spectrum or controlling celestial navigation in the ultraviolet spectrum and for attitude stabilisation purposes. Biologically inspired visual sensors do not have to comprise a very large number of pixels: they are able to perform both short and long range navigation tasks surprisingly well with just a few pixels and a weak resolution.

Agile Robotic Fliers: A Morphing-Based Approach.

The aerial robot presented here for the first time was based on a quadrotor structure, which is capable of unique morphing performances based on an actuated elastic mechanism. Like birds, which are able to negotiate narrow apertures despite their relatively large wingspan, our Quad-Morphing robot was able to pass through a narrow gap at a high forward speed of 2.5 m.s- 1 by swiftly folding up the structure supporting its propellers. A control strategy was developed to deal with the loss of controllability on the roll axis resulting from the folding process, while keeping the robot stable until it has crossed the gap. In addition, a complete recovery procedure was also implemented to stabilize the robot after the unfolding process. A new metric was also used to quantify the gain in terms of the gap-crossing ability in comparison with that observed with classical quadrotors with rigid bodies. The performances of these morphing robots are presented, and experiments performed with a real flying robot passing through a small aperture by reducing its wingspan by 48% are described and discussed.

Role of the light source position in freely falling hoverflies' stabilization performances.

The stabilization of plummeting hoverflies was filmed and analysed in terms of their wingbeat initiation times as well as the crash and stabilization rates. The flies experienced near-weightlessness for a period of time that depended on their ability to counteract the free fall by triggering their wingbeats. In this paradigm, hoverflies' flight stabilization strategies were investigated here for the first time under two different positions of the light source (overhead and bottom lighting). The crash rates were higher in bottom lighting conditions than with top lighting. In addition, adding a texture to the walls reduced the crash rates only in the overhead lighting condition. The position of the lighting also significantly affected both the stabilization rates and the time taken by the flies to stabilize, which decreased and increased under bottom lighting conditions, respectively, whereas textured walls increased the stabilization rates under both lighting conditions. These results support the idea that flies may mainly base their flight control strategy on visual cues and particularly that the light distribution in the visual field may provide reliable, efficient cues for estimating their orientation with respect to an allocentric reference frame. In addition, the finding that the hoverflies' optic flow-based motion detection ability is affected by the position of the light source in their visual field suggests the occurrence of interactions between movement perception and this visual vertical perception process.

Modeling visual-based pitch, lift and speed control strategies in hoverflies.

To avoid crashing onto the floor, a free falling fly needs to trigger its wingbeats quickly and control the orientation of its thrust accurately and swiftly to stabilize its pitch and hence its speed. Behavioural data have suggested that the vertical optic flow produced by the fall and crossing the visual field plays a key role in this anti-crash response. Free fall behavior analyses have also suggested that flying insect may not rely on graviception to stabilize their flight. Based on these two assumptions, we have developed a model which accounts for hoverflies´ position and pitch orientation recorded in 3D with a fast stereo camera during experimental free falls. Our dynamic model shows that optic flow-based control combined with closed-loop control of the pitch suffice to stabilize the flight properly. In addition, our model sheds a new light on the visual-based feedback control of fly´s pitch, lift and thrust. Since graviceptive cues are possibly not used by flying insects, the use of a vertical reference to control the pitch is discussed, based on the results obtained on a complete dynamic model of a virtual fly falling in a textured corridor. This model would provide a useful tool for understanding more clearly how insects may or not estimate their absolute attitude.

Local Positioning System Using Flickering Infrared LEDs.

A minimalistic optical sensing device for the indoor localization is proposed to estimate the relative position between the sensor and active markers using amplitude modulated infrared light. The innovative insect-based sensor can measure azimuth and elevation angles with respect to two small and cheap active infrared light emitting diodes (LEDs) flickering at two different frequencies. In comparison to a previous lensless visual sensor that we proposed for proximal localization (less than 30 cm), we implemented: (i) a minimalistic sensor in terms of small size (10 cm 3 ), light weight (6 g) and low power consumption (0.4 W); (ii) an Arduino-compatible demodulator for fast analog signal processing requiring low computational resources; and (iii) an indoor positioning system for a mobile robotic application. Our results confirmed that the proposed sensor was able to estimate the position at a distance of 2 m with an accuracy as small as 2-cm at a sampling frequency of 100 Hz. Our sensor can be also suitable to be implemented in a position feedback loop for indoor robotic applications in GPS-denied environment.

To crash or not to crash: how do hoverflies cope with free-fall situations and weightlessness?

Insects' aptitude to perform hovering, automatic landing and tracking tasks involves accurately controlling their head and body roll and pitch movements, but how this attitude control depends on an internal estimation of gravity orientation is still an open question. Gravity perception in flying insects has mainly been studied in terms of grounded animals' tactile orientation responses, but it has not yet been established whether hoverflies use gravity perception cues to detect a nearly weightless state at an early stage. Ground-based microgravity simulators provide biologists with useful tools for studying the effects of changes in gravity. However, in view of the cost and the complexity of these set-ups, an alternative Earth-based free-fall procedure was developed with which flying insects can be briefly exposed to microgravity under various visual conditions. Hoverflies frequently initiated wingbeats in response to an imposed free fall in all the conditions tested, but managed to avoid crashing only in variably structured visual environments, and only episodically in darkness. Our results reveal that the crash-avoidance performance of these insects in various visual environments suggests the existence of a multisensory control system based mainly on vision rather than gravity perception.

Behavioural evidence for a visual and proprioceptive control of head roll in hoverflies (Episyrphus balteatus).

The ability of hoverflies to control their head orientation with respect to their body contributes importantly to their agility and their autonomous navigation abilities. Many tasks performed by this insect during flight, especially while hovering, involve a head stabilization reflex. This reflex, which is mediated by multisensory channels, prevents the visual processing from being disturbed by motion blur and maintains a consistent perception of the visual environment. The so-called dorsal light response (DLR) is another head control reflex, which makes insects sensitive to the brightest part of the visual field. In this study, we experimentally validate and quantify the control loop driving the head roll with respect to the horizon in hoverflies. The new approach developed here consisted of using an upside-down horizon in a body roll paradigm. In this unusual configuration, tethered flying hoverflies surprisingly no longer use purely vision-based control for head stabilization. These results shed new light on the role of neck proprioceptor organs in head and body stabilization with respect to the horizon. Based on the responses obtained with male and female hoverflies, an improved model was then developed in which the output signals delivered by the neck proprioceptor organs are combined with the visual error in the estimated position of the body roll. An internal estimation of the body roll angle with respect to the horizon might explain the extremely accurate flight performances achieved by some hovering insects.

An artificial elementary eye with optic flow detection and compositional properties.

We describe a 2 mg artificial elementary eye whose structure and functionality is inspired by compound eye ommatidia. Its optical sensitivity and electronic architecture are sufficient to generate the required signals for the measurement of local optic flow vectors in multiple directions. Multiple elementary eyes can be assembled to create a compound vision system of desired shape and curvature spanning large fields of view. The system configurability is validated with the fabrication of a flexible linear array of artificial elementary eyes capable of extracting optic flow over multiple visual directions.