Real-Time Human Detection and Gesture Recognition for On-Board UAV Rescue.

Unmanned aerial vehicles (UAVs) play an important role in numerous technical and scientific fields, especially in wilderness rescue. This paper carries out work on real-time UAV human detection and recognition of body and hand rescue gestures. We use body-featuring solutions to establish biometric communications, like yolo3-tiny for human detection. When the presence of a person is detected, the system will enter the gesture recognition phase, where the user and the drone can communicate briefly and effectively, avoiding the drawbacks of speech communication. A data-set of ten body rescue gestures (i.e., Kick, Punch, Squat, Stand, Attention, Cancel, Walk, Sit, Direction, and PhoneCall) has been created by a UAV on-board camera. The two most important gestures are the novel dynamic Attention and Cancel which represent the set and reset functions respectively. When the rescue gesture of the human body is recognized as Attention, the drone will gradually approach the user with a larger resolution for hand gesture recognition. The system achieves 99.80% accuracy on testing data in body gesture data-set and 94.71% accuracy on testing data in hand gesture data-set by using the deep learning method. Experiments conducted on real-time UAV cameras confirm our solution can achieve our expected UAV rescue purpose.

KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment.

Deep learning methods for image quality assessment (IQA) are limited due to the small size of existing datasets. Extensive datasets require substantial resources both for generating publishable content and annotating it accurately. We present a systematic and scalable approach to creating KonIQ-10k, the largest IQA dataset to date, consisting of 10,073 quality scored images. It is the first in-the-wild database aiming for ecological validity, concerning the authenticity of distortions, the diversity of content, and quality-related indicators. Through the use of crowdsourcing, we obtained 1.2 million reliable quality ratings from 1,459 crowd workers, paving the way for more general IQA models. We propose a novel, deep learning model (KonCept512), to show an excellent generalization beyond the test set (0.921 SROCC), to the current state-of-the-art database LIVE-in-the-Wild (0.825 SROCC). The model derives its core performance from the InceptionResNet architecture, being trained at a higher resolution than previous models (512 × 384). Correlation analysis shows that KonCept512 performs similar to having 9 subjective scores for each test image.

Adaptive image decomposition into cartoon and texture parts optimized by the orthogonality criterion.

In this paper a new decomposition method is introduced that splits the image into geometric (or cartoon) and texture parts. Following a total variation based preprocesssing, the core of the proposed method is an anisotropic diffusion with an orthogonality based parameter estimation and stopping condition. The quality criterion is defined by the theoretical assumption that the cartoon and the texture components of an image should be orthogonal to each other. The presented method has been compared to other decomposition algorithms through visual and numerical evaluation to prove its superiority.

Detection of object motion regions in aerial image pairs with a multilayer markovian model.

We propose a new Bayesian method for detecting the regions of object displacements in aerial image pairs. We use a robust but coarse 2-D image registration algorithm. Our main challenge is to eliminate the registration errors from the extracted change map. We introduce a three-layer Markov random field (L(3)MRF) model which integrates information from two different features, and ensures connected homogenous regions in the segmented images. Validation is given on real aerial photos.

The use of vanishing point for the classification of reflections from foreground mask in videos.

Extraction of foreground is a basic task in surveillance video analysis. In most real cases, its performance is heavily based on the efficiency of shadow detection and on the analysis of lighting conditions and reflections caused by mirrors or other reflective surfaces. This correspondence is focused on the improvement of foreground extraction in the case of planar reflective surfaces. We show that the geometric model of a scene with a planar reflective surface is reduced to the estimation of vanishing-point for the case of an auto-epipolar (skew-symmetric) fundamental matrix. The correspondences for the vanishing-point estimation are extracted from motion statistics. The knowledge of the position of the vanishing point allows us to integrate the geometric model and the motion statistics into image foreground-extraction to separate foreground from reflections, and thus to achieve better performance. The experiments confirm the accuracy of the vanishing point and the improvement of the foreground image mask by removing reflected object parts.

Focus area extraction by blind deconvolution for defining regions of interest.

We present an automatic focus area estimation method, working with a single image without a priori information about the image, the camera, or the scene. It produces relative focus maps by localized blind deconvolution and a new residual error-based classification. Evaluation and comparison is performed and applicability is shown through image indexing.

Stochastic view registration of overlapping cameras based on arbitrary motion.

A new motion-based method is presented for automatic registration of images in multicamera systems, to permit synthesis of wide-baseline composite views. Unlike existing static-image and motion-based methods, our approach does not need any a priori information about the scene, the appearance of objects in the scene, or their motion. We introduce an entropy-based preselection of motion histories and an iterative Bayesian assignment of corresponding image areas. Finally, correlated point-histories and data-set optimization lead to the matching of the different views. The method is validated by demonstrating its successful use on several real-life indoor and outdoor stereo video image-sequence pairs.

Detection of gait characteristics for scene registration in video surveillance system.

This paper presents a robust walk-detection algorithm, based on our symmetry approach which can be used to extract gait characteristics from video-image sequences. To obtain a useful descriptor of a walking person, we temporally track the symmetries of a person's legs. Our method is suitable for use in indoor or outdoor surveillance scenes. Determining the leading leg of the walking subject is important, and the presented method can identify this from two successive walk steps (one walk cycle). We tested the accuracy of the presented walk-detection method in a possible application: Image registration methods are presented which are applicable to multicamera systems viewing human subjects in motion.

Estimation of the vanishing point in camera-mirror scenes from video.

Knowledge of the position of the vanishing point is the key for geometrical modeling of an image containing a reflective surface or cast shadows. Such an image can be analyzed as two subimages that constitute a stereo pair. For this model-estimation task an automatic method is presented that utilizes motion statistics and the statistical properties of image points for the determination of point correspondence and the subsequent estimation of vanishing point position, optimized by use of a goodness-of-fit function. We show that this approach gives robust results in widely different real-world environments, even when the correspondence is corrupted with considerable amounts of noise.

Relative focus map estimation using blind deconvolution.

An automatic focus map extraction method is presented that uses a modification of blind deconvolution for estimation of localized blurring functions. We use these local blurring functions [so-called point-spread functions (PSFs)] for extraction of focus areas on ordinary images. In this inverse task our goal is not image reconstruction but the estimation of localized PSFs and the relative focus map. Thus the method is less sensitive than general deconvolution is to noise and ill-posed deconvolution problems. The focus areas can be estimated without any knowledge of the shooting conditions or of the optical system used.