Neural Disparity Refinement.

We propose a framework that combines traditional, hand-crafted algorithms and recent advances in deep learning to obtain high-quality, high-resolution disparity maps from stereo images. By casting the refinement process as a continuous feature sampling strategy, our neural disparity refinement network can estimate an enhanced disparity map at any output resolution. Our solution can process any disparity map produced by classical stereo algorithms, as well as those predicted by modern stereo networks or even different depth-from-images approaches, such as the COLMAP structure-from-motion pipeline. Nonetheless, when deployed in the former configuration, our framework performs at its best in terms of zero-shot generalization from synthetic to real images. Moreover, its continuous formulation allows for easily handling the unbalanced stereo setup very diffused in mobile phones.

Self-supervised depth super-resolution with contrastive multiview pre-training.

Many low-level vision tasks, including guided depth super-resolution (GDSR), struggle with the issue of insufficient paired training data. Self-supervised learning is a promising solution, but it remains challenging to upsample depth maps without the explicit supervision of high-resolution target images. To alleviate this problem, we propose a self-supervised depth super-resolution method with contrastive multiview pre-training. Unlike existing contrastive learning methods for classification or segmentation tasks, our strategy can be applied to regression tasks even when trained on a small-scale dataset and can reduce information redundancy by extracting unique features from the guide. Furthermore, we propose a novel mutual modulation scheme that can effectively compute the local spatial correlation between cross-modal features. Exhaustive experiments demonstrate that our method attains superior performance with respect to state-of-the-art GDSR methods and exhibits good generalization to other modalities.

On the Confidence of Stereo Matching in a Deep-Learning Era: A Quantitative Evaluation.

Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images. Alongside with the development of more accurate algorithms, the research community focused on finding good strategies to estimate the reliability, i.e., the confidence, of estimated disparity maps. This information proves to be a powerful cue to naively find wrong matches as well as to improve the overall effectiveness of a variety of stereo algorithms according to different strategies. In this paper, we review more than ten years of developments in the field of confidence estimation for stereo matching. We extensively discuss and evaluate existing confidence measures and their variants, from hand-crafted ones to the most recent, state-of-the-art learning based methods. We study the different behaviors of each measure when applied to a pool of different stereo algorithms and, for the first time in literature, when paired with a state-of-the-art deep stereo network. Our experiments, carried out on five different standard datasets, provide a comprehensive overview of the field, highlighting in particular both strengths and limitations of learning-based strategies.

On the Synergies Between Machine Learning and Binocular Stereo for Depth Estimation From Images: A Survey.

Stereo matching is one of the longest-standing problems in computer vision with close to 40 years of studies and research. Throughout the years the paradigm has shifted from local, pixel-level decision to various forms of discrete and continuous optimization to data-driven, learning-based methods. Recently, the rise of machine learning and the rapid proliferation of deep learning enhanced stereo matching with new exciting trends and applications unthinkable until a few years ago. Interestingly, the relationship between these two worlds is two-way. While machine, and especially deep, learning advanced the state-of-the-art in stereo matching, stereo itself enabled new ground-breaking methodologies such as self-supervised monocular depth estimation based on deep networks. In this paper, we review recent research in the field of learning-based depth estimation from single and binocular images highlighting the synergies, the successes achieved so far and the open challenges the community is going to face in the immediate future.

Continual Adaptation for Deep Stereo.

Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applications, the capacity of adapting to any unseen setting becomes of paramount importance. Purposely, we propose a continual adaptation paradigm for deep stereo networks designed to deal with challenging and ever-changing environments. We design a lightweight and modular architecture, Modularly ADaptive Network (MADNet), and formulate Modular ADaptation algorithms (MAD, MAD++) which permit efficient optimization of independent sub-portions of the entire network. In our paradigm, the learning signals needed to continuously adapt models online can be sourced from self-supervision via right-to-left image warping or from traditional stereo algorithms. With both sources, no other data than the input images being gathered at deployment time are needed. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate practical deployment of end-to-end architectures for dense disparity regression.

Real-Time Single Image Depth Perception in the Wild with Handheld Devices.

Depth perception is paramount for tackling real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image would represent the most versatile solution since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit the practical deployment of monocular depth estimation methods on such devices: (i) the low reliability when deployed in the wild and (ii) the resources needed to achieve real-time performance, often not compatible with low-power embedded systems. Therefore, in this paper, we deeply investigate all these issues, showing how they are both addressable by adopting appropriate network design and training strategies. Moreover, we also outline how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time, depth-aware augmented reality and image blurring with smartphones in the wild.

May a unitary autonomic index help assess autonomic cardiac regulation in elite athletes? Preliminary observations on the national Italian Olympic committee team.

BACKGROUND: Long term endurance training, as occurring in elite athletes, is associated to cardiac neural remodeling in favor of cardioprotective vagal mechanisms, resulting in resting bradycardia and augmented contribution of cardiac parasympathetic nerve activity. Autonomic assessment can be performed by way of heart rate variability. This technique however provides multiple indices, and there is not yet complete agreement on their specific significance. Purpose of the study was to assess whether a rank transformation and radar plot could provide a unitary autonomic index, capable to show a correlation between intensity of individual work and quality of autonomic regulation. METHODS: We studied 711 (23.6±6.2 years) elite athletes that took part in the selection procedure for the 2016 Rio Olympic Games for the National Italian Olympic Committee (CONI). Indices from Heart Rate Variability HRV obtained at rest, during standing up and during recovery from an exercise test were used to compute a percent ranked unitary autonomic index for sport (ANSIs), taken as proxy of quality of autonomic regulation. RESULTS: Within the observed wide range of energy expenditure, the unitary autonomic index ANSIs appears significantly correlated to individual and discipline specific training workloads (r=0.25, P<0.001 and r=0.78, P<0.001, respectively), correcting for possible age and gender bias. ANSIs also positively correlates to lipid profile. CONCLUSIONS: Estimated intensity of physical activity correlates with quality of cardiac autonomic regulation, as expressed by a novel unitary index of cardiac autonomic regulation. ANSIs could provide a novel and convenient approach to individual autonomic evaluation in athletes.

Assessing autonomic response to repeated bouts of exercise below and above respiratory threshold: insight from dynamic analysis of RR variability.

PURPOSE: The dynamics of vagal withdrawal and reactivation during pulses of exercise are described by indices computed from heart period (RR) variations, which may be sensitive to duration and load. We sought to assess the consistency over time of these indices, which is not well established. METHODS: We recorded continuous electrocardiogram during series of five successive bouts (2 min) of submaximal exercise (at 40 and 70% of VO(2peak), different days). Autonomic responsiveness was inferred from quantification of onset and offset of RR dynamics of each individual bout. Consistency of results was assessed with intraclass correlation (ICC). RESULTS: During exercise bouts, indices from tachycardic and bradycardic transients reach lower levels in response to higher exercise loads and progression of exercise. There is a significant effect of load and time (i.e., bout repetition) for all examined variables, with a clear interaction. However, no interaction is observed with the 60 s change in heart rate. ICC analysis demonstrates that various indices are characterized by large differences in stability, which is generally greater within the same day (e.g., tachyspeed ICC at 40% = 0.751, at 70% = 0.704, both days = 0.633; bradyspeed, respectively, = 0.545, 0.666, 0.516). CONCLUSIONS: Intensity and duration of exercise modulate vagal withdrawal and reactivation. Analysis of RR variations, during successive brief exercise bouts at lower and higher intensity, ensures a consistency similar to that reported for autonomic cardiac regulation at rest and might guide the choice among multiple indices that are obtained from the tachogram.

Altered cardiovascular autonomic regulation in overweight children engaged in regular physical activity.

Overweight (OW) and obesity in children are important forerunners of cardiovascular risk, possibly through autonomic nervous system (ANS) dysregulation, while physical exercise exerts a beneficial influence. In this observational study we hypothesise that OW might influence ANS profile even in a population performing high volume of supervised exercise. We study 103 young soccer players, homogeneous in terms of gender (all male), cultural background, school, age (11.2 ± 1 years) and exercise routine, since they all belong to the same soccer club, thus guaranteeing equality of supervised training and similar levels of competitiveness. ANS is evaluated by autoregressive spectral analysis of heart rate and systolic arterial pressure (SAP) variabilities. We estimate also the accumulated weekly Metabolic Equivalents and time spent in sedentary activities. We subdivide the entire population in two subgroups (normal weight and OW) based on the International Obesity Task Force criteria. In OW soccer players (10.7% of total group) we observe an altered profile of autonomic cardiovascular regulation, characterised by higher values of SAP (113 ± 4 vs 100 ± 1 mm Hg, 39.7 ± 3 vs 66.2 ± 10%), higher Low Frequency variability power of SAP (an index of vasomotor sympathetic regulation) (12 ± 3 vs 4.5 mm Hg(2)) and smaller spontaneous baroreflex gain (an index of cardiac vagal regulation) (19 ± 3 vs 33 ± 3 ms/mm Hg) (all (p < 0.02)). Moreover Correlation analysis on the entire study population shows a significant link between anthropometric and autonomic indices. These data show that OW is associated to a clear autonomic impairment even in children subjected to an intense aerobic training.