Mask R-CNN.

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each instance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., allowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without bells and whistles, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code has been made available at: https://github.com/facebookresearch/Detectron.

Focal Loss for Dense Object Detection.

The highest accuracy object detectors to date are based on a two-stage approach popularized by R-CNN, where a classifier is applied to a sparse set of candidate object locations. In contrast, one-stage detectors that are applied over a regular, dense sampling of possible object locations have the potential to be faster and simpler, but have trailed the accuracy of two-stage detectors thus far. In this paper, we investigate why this is the case. We discover that the extreme foreground-background class imbalance encountered during training of dense detectors is the central cause. We propose to address this class imbalance by reshaping the standard cross entropy loss such that it down-weights the loss assigned to well-classified examples. Our novel Focal Loss focuses training on a sparse set of hard examples and prevents the vast number of easy negatives from overwhelming the detector during training. To evaluate the effectiveness of our loss, we design and train a simple dense detector we call RetinaNet. Our results show that when trained with the focal loss, RetinaNet is able to match the speed of previous one-stage detectors while surpassing the accuracy of all existing state-of-the-art two-stage detectors. Code is at: https://github.com/facebookresearch/Detectron.

What Makes for Effective Detection Proposals?

Current top performing object detectors employ detection proposals to guide the search for objects, thereby avoiding exhaustive sliding window search across images. Despite the popularity and widespread use of detection proposals, it is unclear which trade-offs are made when using them during object detection. We provide an in-depth analysis of twelve proposal methods along with four baselines regarding proposal repeatability, ground truth annotation recall on PASCAL, ImageNet, and MS COCO, and their impact on DPM, R-CNN, and Fast R-CNN detection performance. Our analysis shows that for object detection improving proposal localisation accuracy is as important as improving recall. We introduce a novel metric, the average recall (AR), which rewards both high recall and good localisation and correlates surprisingly well with detection performance. Our findings show common strengths and weaknesses of existing methods, and provide insights and metrics for selecting and tuning proposal methods.

Fast Edge Detection Using Structured Forests.

Edge detection is a critical component of many vision systems, including object detectors and image segmentation algorithms. Patches of edges exhibit well-known forms of local structure, such as straight lines or T-junctions. In this paper we take advantage of the structure present in local image patches to learn both an accurate and computationally efficient edge detector. We formulate the problem of predicting local edge masks in a structured learning framework applied to random decision forests. Our novel approach to learning decision trees robustly maps the structured labels to a discrete space on which standard information gain measures may be evaluated. The result is an approach that obtains realtime performance that is orders of magnitude faster than many competing state-of-the-art approaches, while also achieving state-of-the-art edge detection results on the BSDS500 Segmentation dataset and NYU Depth dataset. Finally, we show the potential of our approach as a general purpose edge detector by showing our learned edge models generalize well across datasets.

Decoding ventromedial hypothalamic neural activity during male mouse aggression.

The ventromedial hypothalamus, ventrolateral area (VMHvl) was identified recently as a critical locus for inter-male aggression. Optogenetic stimulation of VMHvl in male mice evokes attack toward conspecifics and inactivation of the region inhibits natural aggression, yet very little is known about its underlying neural activity. To understand its role in promoting aggression, we recorded and analyzed neural activity in the VMHvl in response to a wide range of social and nonsocial stimuli. Although response profiles of VMHvl neurons are complex and heterogeneous, we identified a subpopulation of neurons that respond maximally during investigation and attack of male conspecific mice and during investigation of a source of male mouse urine. These "male responsive" neurons in the VMHvl are tuned to both the inter-male distance and the animal's velocity during attack. Additionally, VMHvl activity predicts several parameters of future aggressive action, including the latency and duration of the next attack. Linear regression analysis further demonstrates that aggression-specific parameters, such as distance, movement velocity, and attack latency, can model ongoing VMHvl activity fluctuation during inter-male encounters. These results represent the first effort to understand the hypothalamic neural activity during social behaviors using quantitative tools and suggest an important role for the VMHvl in encoding movement, sensory, and motivation-related signals.

Fast Feature Pyramids for Object Detection.

Multi-resolution image features may be approximated via extrapolation from nearby scales, rather than being computed explicitly. This fundamental insight allows us to design object detection algorithms that are as accurate, and considerably faster, than the state-of-the-art. The computational bottleneck of many modern detectors is the computation of features at every scale of a finely-sampled image pyramid. Our key insight is that one may compute finely sampled feature pyramids at a fraction of the cost, without sacrificing performance: for a broad family of features we find that features computed at octave-spaced scale intervals are sufficient to approximate features on a finely-sampled pyramid. Extrapolation is inexpensive as compared to direct feature computation. As a result, our approximation yields considerable speedups with negligible loss in detection accuracy. We modify three diverse visual recognition systems to use fast feature pyramids and show results on both pedestrian detection (measured on the Caltech, INRIA, TUD-Brussels and ETH data sets) and general object detection (measured on the PASCAL VOC). The approach is general and is widely applicable to vision algorithms requiring fine-grained multi-scale analysis. Our approximation is valid for images with broad spectra (most natural images) and fails for images with narrow band-pass spectra (e.g., periodic textures).

Pedestrian detection: an evaluation of the state of the art.

Pedestrian detection is a key problem in computer vision, with several applications that have the potential to positively impact quality of life. In recent years, the number of approaches to detecting pedestrians in monocular images has grown steadily. However, multiple data sets and widely varying evaluation protocols are used, making direct comparisons difficult. To address these shortcomings, we perform an extensive evaluation of the state of the art in a unified framework. We make three primary contributions: 1) We put together a large, well-annotated, and realistic monocular pedestrian detection data set and study the statistics of the size, position, and occlusion patterns of pedestrians in urban scenes, 2) we propose a refined per-frame evaluation methodology that allows us to carry out probing and informative comparisons, including measuring performance in relation to scale and occlusion, and 3) we evaluate the performance of sixteen pretrained state-of-the-art detectors across six data sets. Our study allows us to assess the state of the art and provides a framework for gauging future efforts. Our experiments show that despite significant progress, performance still has much room for improvement. In particular, detection is disappointing at low resolutions and for partially occluded pedestrians.

Functional identification of an aggression locus in the mouse hypothalamus.

Electrical stimulation of certain hypothalamic regions in cats and rodents can elicit attack behaviour, but the exact location of relevant cells within these regions, their requirement for naturally occurring aggression and their relationship to mating circuits have not been clear. Genetic methods for neural circuit manipulation in mice provide a potentially powerful approach to this problem, but brain-stimulation-evoked aggression has never been demonstrated in this species. Here we show that optogenetic, but not electrical, stimulation of neurons in the ventromedial hypothalamus, ventrolateral subdivision (VMHvl) causes male mice to attack both females and inanimate objects, as well as males. Pharmacogenetic silencing of VMHvl reversibly inhibits inter-male aggression. Immediate early gene analysis and single unit recordings from VMHvl during social interactions reveal overlapping but distinct neuronal subpopulations involved in fighting and mating. Neurons activated during attack are inhibited during mating, suggesting a potential neural substrate for competition between these opponent social behaviours.

Daily timed sexual interaction induces moderate anticipatory activity in mice.

Anticipation of resource availability is a vital skill yet it is poorly understood in terms of neuronal circuitry. Rodents display robust anticipatory activity in the several hours preceding timed daily access to food when access is limited to a short temporal duration. We tested whether this anticipatory behavior could be generalized to timed daily social interaction by examining if singly housed male mice could anticipate either a daily novel female or a familiar female. We observed that anticipatory activity was moderate under both conditions, although both a novel female partner and sexual experience are moderate contributing factors to increasing anticipatory activity. In contrast, restricted access to running wheels did not produce any anticipatory activity, suggesting that an increase in activity during the scheduled access time was not sufficient to induce anticipation. To tease apart social versus sexual interaction, we tested the effect of exposing singly housed female mice to a familiar companion female mouse daily. The female mice did not show anticipatory activity for restricted female access, despite a large amount of social interaction, suggesting that daily timed social interaction between mice of the same gender is insufficient to induce anticipatory activity. Our study demonstrates that male mice will show anticipatory activity, albeit inconsistently, for a daily timed sexual encounter.