Analytical Bounds on the Local Lipschitz Constants of ReLU Networks.

In this article, we determine analytical upper bounds on the local Lipschitz constants of feedforward neural networks with rectified linear unit (ReLU) activation functions. We do so by deriving Lipschitz constants and bounds for ReLU, affine-ReLU, and max-pooling functions and combining the results to determine a network-wide bound. Our method uses several insights to obtain tight bounds, such as keeping track of the zero elements of each layer and analyzing the composition of affine and ReLU functions. Furthermore, we employ a careful computational approach which allows us to apply our method to large networks, such as AlexNet and VGG-16. We present several examples using different networks, which show how our local Lipschitz bounds are tighter than the global Lipschitz bounds. We also show how our method can be applied to provide adversarial bounds for classification networks. These results show that our method produces the largest known bounds on minimum adversarial perturbations for large networks, such as AlexNet and VGG-16.

Using collision cones to assess biological deconfliction methods.

Biological systems consistently outperform autonomous systems governed by engineered algorithms in their ability to reactively avoid collisions. To better understand this discrepancy, a collision avoidance algorithm was applied to frames of digitized video trajectory data from bats, swallows and fish (Myotis velifer, Petrochelidon pyrrhonota and Danio aequipinnatus). Information available from visual cues, specifically relative position and velocity, was provided to the algorithm which used this information to define collision cones that allowed the algorithm to find a safe velocity requiring minimal deviation from the original velocity. The subset of obstacles provided to the algorithm was determined by the animal's sensing range in terms of metric and topological distance. The algorithmic calculated velocities showed good agreement with observed biological velocities, indicating that the algorithm was an informative basis for comparison with the three species and could potentially be improved for engineered applications with further study.

Gyroscopic sensing in the wings of the hawkmoth Manduca sexta: the role of sensor location and directional sensitivity.

The wings of the hawkmoth Manduca sexta are lined with mechanoreceptors called campaniform sensilla that encode wing deformations. During flight, the wings deform in response to a variety of stimuli, including inertial-elastic loads due to the wing flapping motion, aerodynamic loads, and exogenous inertial loads transmitted by disturbances. Because the wings are actuated, flexible structures, the strain-sensitive campaniform sensilla are capable of detecting inertial rotations and accelerations, allowing the wings to serve not only as a primary actuator, but also as a gyroscopic sensor for flight control. We study the gyroscopic sensing of the hawkmoth wings from a control theoretic perspective. Through the development of a low-order model of flexible wing flapping dynamics, and the use of nonlinear observability analysis, we show that the rotational acceleration inherent in wing flapping enables the wings to serve as gyroscopic sensors. We compute a measure of sensor fitness as a function of sensor location and directional sensitivity by using the simulation-based empirical observability Gramian. Our results indicate that gyroscopic information is encoded primarily through shear strain due to wing twisting, where inertial rotations cause detectable changes in pronation and supination timing and magnitude. We solve an observability-based optimal sensor placement problem to find the optimal configuration of strain sensor locations and directional sensitivities for detecting inertial rotations. The optimal sensor configuration shows parallels to the campaniform sensilla found on hawkmoth wings, with clusters of sensors near the wing root and wing tip. The optimal spatial distribution of strain directional sensitivity provides a hypothesis for how heterogeneity of campaniform sensilla may be distributed.

Flexible strategies for flight control: an active role for the abdomen.

Moving animals orchestrate myriad motor systems in response to multimodal sensory inputs. Coordinating movement is particularly challenging in flight control, where animals deal with potential instability and multiple degrees of freedom of movement. Prior studies have focused on wings as the primary flight control structures, for which changes in angle of attack or shape are used to modulate lift and drag forces. However, other actuators that may impact flight performance are reflexively activated during flight. We investigated the visual-abdominal reflex displayed by the hawkmoth Manduca sexta to determine its role in flight control. We measured the open-loop stimulus-response characteristics (measured as a transfer function) between the visual stimulus and abdominal response in tethered moths. The transfer function reveals a 41 ms delay and a high-pass filter behavior with a pass band starting at ~0.5 Hz. We also developed a simplified mathematical model of hovering flight wherein articulation of the thoracic-abdominal joint redirects an average lift force provided by the wings. We show that control of the joint, subject to a high-pass filter, is sufficient to maintain stable hovering, but with a slim stability margin. Our experiments and models suggest a novel mechanism by which articulation of the body or 'airframe' of an animal can be used to redirect lift forces for effective flight control. Furthermore, the small stability margin may increase flight agility by easing the transition from stable flight to a more maneuverable, unstable regime.