Application-Oriented Retinal Image Models for Computer Vision.

Energy and storage restrictions are relevant variables that software applications should be concerned about when running in low-power environments. In particular, computer vision (CV) applications exemplify well that concern, since conventional uniform image sensors typically capture large amounts of data to be further handled by the appropriate CV algorithms. Moreover, much of the acquired data are often redundant and outside of the application's interest, which leads to unnecessary processing and energy spending. In the literature, techniques for sensing and re-sampling images in non-uniform fashions have emerged to cope with these problems. In this study, we propose Application-Oriented Retinal Image Models that define a space-variant configuration of uniform images and contemplate requirements of energy consumption and storage footprints for CV applications. We hypothesize that our models might decrease energy consumption in CV tasks. Moreover, we show how to create the models and validate their use in a face detection/recognition application, evidencing the compromise between storage, energy, and accuracy.

Weak Classifier for Density Estimation in Eye Localization and Tracking.

In this paper, we propose a fast weak classifier that can detect and track eyes in video sequences. The approach relies on a least-squares detector based on the inner product detector (IPD) that can stimate a probability density distribution for a feature's location-which fits naturally with a Bayesian estimation cycle, such as a Kalman or particle filter. As a least-squares sliding window detector, it possesses tolerance to small variations in the desired pattern while maintaining good generalization capabilities and computational efficiency. We propose two approaches to integrating the IPD with a particle filter tracker. We use the BioID, FERET, LFPW, and COFW public datasets as well as five manually annotated high-definition video sequences to quantitatively evaluate the algorithms' performance. The video data set contains four subjects, different types of backgrounds, blurring due to fast motion, and occlusions. All code and data are available.

PSQP: Puzzle Solving by Quadratic Programming.

In this article we present the first effective method based on global optimization for the reconstruction of image puzzles comprising rectangle pieces-Puzzle Solving by Quadratic Programming (PSQP). The proposed novel mathematical formulation reduces the problem to the maximization of a constrained quadratic function, which is solved via a gradient ascent approach. The proposed method is deterministic and can deal with arbitrary identical rectangular pieces. We provide experimental results showing its effectiveness when compared to state-of-the-art approaches. Although the method was developed to solve image puzzles, we also show how to apply it to the reconstruction of simulated strip-shredded documents, broadening its applicability.

On the Reconstruction of Text Phylogeny Trees: Evaluation and Analysis of Textual Relationships.

Over the history of mankind, textual records change. Sometimes due to mistakes during transcription, sometimes on purpose, as a way to rewrite facts and reinterpret history. There are several classical cases, such as the logarithmic tables, and the transmission of antique and medieval scholarship. Today, text documents are largely edited and redistributed on the Web. Articles on news portals and collaborative platforms (such as Wikipedia), source code, posts on social networks, and even scientific publications or literary works are some examples in which textual content can be subject to changes in an evolutionary process. In this scenario, given a set of near-duplicate documents, it is worthwhile to find which one is the original and the history of changes that created the whole set. Such functionality would have immediate applications on news tracking services, detection of plagiarism, textual criticism, and copyright enforcement, for instance. However, this is not an easy task, as textual features pointing to the documents' evolutionary direction may not be evident and are often dataset dependent. Moreover, side information, such as time stamps, are neither always available nor reliable. In this paper, we propose a framework for reliably reconstructing text phylogeny trees, and seamlessly exploring new approaches on a wide range of scenarios of text reusage. We employ and evaluate distinct combinations of dissimilarity measures and reconstruction strategies within the proposed framework, and evaluate each approach with extensive experiments, including a set of artificial near-duplicate documents with known phylogeny, and from documents collected from Wikipedia, whose modifications were made by Internet users. We also present results from qualitative experiments in two different applications: text plagiarism and reconstruction of evolutionary trees for manuscripts (stemmatology).

Pornography classification: The hidden clues in video space-time.

As web technologies and social networks become part of the general public's life, the problem of automatically detecting pornography is into every parent's mind - nobody feels completely safe when their children go online. In this paper, we focus on video-pornography classification, a hard problem in which traditional methods often employ still-image techniques - labeling frames individually prior to a global decision. Frame-based approaches, however, ignore significant cogent information brought by motion. Here, we introduce a space-temporal interest point detector and descriptor called Temporal Robust Features (TRoF). TRoF was custom-tailored for efficient (low processing time and memory footprint) and effective (high classification accuracy and low false negative rate) motion description, particularly suited to the task at hand. We aggregate local information extracted by TRoF into a mid-level representation using Fisher Vectors, the state-of-the-art model of Bags of Visual Words (BoVW). We evaluate our original strategy, contrasting it both to commercial pornography detection solutions, and to BoVW solutions based upon other space-temporal features from the scientific literature. The performance is assessed using the Pornography-2k dataset, a new challenging pornographic benchmark, comprising 2000 web videos and 140h of video footage. The dataset is also a contribution of this work and is very assorted, including both professional and amateur content, and it depicts several genres of pornography, from cartoon to live action, with diverse behavior and ethnicity. The best approach, based on a dense application of TRoF, yields a classification error reduction of almost 79% when compared to the best commercial classifier. A sparse description relying on TRoF detector is also noteworthy, for yielding a classification error reduction of over 69%, with 19× less memory footprint than the dense solution, and yet can also be implemented to meet real-time requirements.

Automated multi-lesion detection for referable diabetic retinopathy in indigenous health care.

Diabetic Retinopathy (DR) is a complication of diabetes mellitus that affects more than one-quarter of the population with diabetes, and can lead to blindness if not discovered in time. An automated screening enables the identification of patients who need further medical attention. This study aimed to classify retinal images of Aboriginal and Torres Strait Islander peoples utilizing an automated computer-based multi-lesion eye screening program for diabetic retinopathy. The multi-lesion classifier was trained on 1,014 images from the São Paulo Eye Hospital and tested on retinal images containing no DR-related lesion, single lesions, or multiple types of lesions from the Inala Aboriginal and Torres Strait Islander health care centre. The automated multi-lesion classifier has the potential to enhance the efficiency of clinical practice delivering diabetic retinopathy screening. Our program does not necessitate image samples for training from any specific ethnic group or population being assessed and is independent of image pre- or post-processing to identify retinal lesions. In this Aboriginal and Torres Strait Islander population, the program achieved 100% sensitivity and 88.9% specificity in identifying bright lesions, while detection of red lesions achieved a sensitivity of 67% and specificity of 95%. When both bright and red lesions were present, 100% sensitivity with 88.9% specificity was obtained. All results obtained with this automated screening program meet WHO standards for diabetic retinopathy screening.

Multiclass from binary: expanding one-versus-all, one-versus-one and ECOC-based approaches.

Recently, there has been a lot of success in the development of effective binary classifiers. Although many statistical classification techniques have natural multiclass extensions, some, such as the support vector machines, do not. The existing techniques for mapping multiclass problems onto a set of simpler binary classification problems run into serious efficiency problems when there are hundreds or even thousands of classes, and these are the scenarios where this paper's contributions shine. We introduce the concept of correlation and joint probability of base binary learners. We learn these properties during the training stage, group the binary leaner's based on their independence and, with a Bayesian approach, combine the results to predict the class of a new instance. Finally, we also discuss two additional strategies: one to reduce the number of required base learners in the multiclass classification, and another to find new base learners that might best complement the existing set. We use these two new procedures iteratively to complement the initial solution and improve the overall performance. This paper has two goals: finding the most discriminative binary classifiers to solve a multiclass problem and keeping up the efficiency, i.e., small number of base learners. We validate and compare the method with a diverse set of methods of the literature in several public available datasets that range from small (10 to 26 classes) to large multiclass problems (1000 classes) always using simple reproducible scenarios.

Assessing the Need for Referral in Automatic Diabetic Retinopathy Detection.

Emerging technologies in health care aim at reducing unnecessary visits to medical specialists, minimizing overall cost of treatment and optimizing the number of patients seen by each doctor. This paper explores image recognition for the screening of diabetic retinopathy, a complication of diabetes that can lead to blindness if not discovered in its initial stages. Many previous reports on DR imaging focus on the segmentation of the retinal image, on quality assessment, and on the analysis of presence of DR-related lesions. Although this study has advanced the detection of individual DR lesions from retinal images, the simple presence of any lesion is not enough to decide on the need for referral of a patient. Deciding if a patient should be referred to a doctor is an essential requirement for the deployment of an automated screening tool for rural and remote communities. We introduce an algorithm to make that decision based on the fusion of results by metaclassification. The input of the metaclassifier is the output of several lesion detectors, creating a powerful high-level feature representation for the retinal images. We explore alternatives for the bag-of-visual-words (BoVW)-based lesion detectors, which critically depends on the choices of coding and pooling the low-level local descriptors. The final classification approach achieved an area under the curve of 93.4% using SOFT-MAX BoVW (soft-assignment coding/max pooling), without the need of normalizing the high-level feature vector of scores.

Toward image phylogeny forests: automatically recovering semantically similar image relationships.

In the past few years, several near-duplicate detection methods appeared in the literature to identify the cohabiting versions of a given document online. Following this trend, there are some initial attempts to go beyond the detection task, and look into the structure of evolution within a set of related images overtime. In this paper, we aim at automatically identify the structure of relationships underlying the images, correctly reconstruct their past history and ancestry information, and group them in distinct trees of processing history. We introduce a new algorithm that automatically handles sets of images comprising different related images, and outputs the phylogeny trees (also known as a forest) associated with them. Image phylogeny algorithms have many applications such as finding the first image within a set posted online (useful for tracking copyright infringement perpetrators), hint at child pornography content creators, and narrowing down a list of suspects for online harassment using photographs.

Machine learning and pattern classification in identification of indigenous retinal pathology.

Diabetic retinopathy (DR) is a complication of diabetes, which if untreated leads to blindness. DR early diagnosis and treatment improve outcomes. Automated assessment of single lesions associated with DR has been investigated for sometime. To improve on classification, especially across different ethnic groups, we present an approach using points-of-interest and visual dictionary that contains important features required to identify retinal pathology. Variation in images of the human retina with respect to differences in pigmentation and presence of diverse lesions can be analyzed without the necessity of preprocessing and utilizing different training sets to account for ethnic differences for instance.

Measuring handball players trajectories using an automatically trained boosting algorithm.

The aim of the present paper is to propose and evaluate an automatically trained cascaded boosting detector algorithm based on morphological segmentation for tracking handball players. The proposed method was able to detect correctly 84% of players when applied to the second period of that same game used for training and 74% when applied to a different game. Furthermore, the analysis of the automatic training using boosting detector revealed general results such as the training time initially increased with the number of figures used, but as more figures were added, the training time decreased. Automatic morphological segmentation has shown to be a fast and efficient method for selecting image regions for the boosting detector and allowed an improvement in the automatic tracking of handball players.

Optical computer recognition of facial expressions associated with stress induced by performance demands.

Application of computer vision to track changes in human facial expressions during long-duration spaceflight may be a useful way to unobtrusively detect the presence of stress during critical operations. To develop such an approach, we applied optical computer recognition (OCR) algorithms for detecting facial changes during performance while people experienced both low- and high-stressor performance demands. Workload and social feedback were used to vary performance stress in 60 healthy adults (29 men, 31 women; mean age 30 yr). High-stressor scenarios involved more difficult performance tasks, negative social feedback, and greater time pressure relative to low workload scenarios. Stress reactions were tracked using self-report ratings, salivary cortisol, and heart rate. Subjects also completed personality, mood, and alexithymia questionnaires. To bootstrap development of the OCR algorithm, we had a human observer, blind to stressor condition, identify the expressive elements of the face of people undergoing high- vs. low-stressor performance. Different sets of videos of subjects' faces during performance conditions were used for OCR algorithm training. Subjective ratings of stress, task difficulty, effort required, frustration, and negative mood were significantly increased during high-stressor performance bouts relative to low-stressor bouts (all p < 0.01). The OCR algorithm was refined to provide robust 3-d tracking of facial expressions during head movement. Movements of eyebrows and asymmetries in the mouth were extracted. These parameters are being used in a Hidden Markov model to identify high- and low-stressor conditions. Preliminary results suggest that an OCR algorithm using mouth and eyebrow regions has the potential to discriminate high- from low-stressor performance bouts in 75-88% of subjects. The validity of the workload paradigm to induce differential levels of stress in facial expressions was established. The paradigm also provided the basic stress-related facial expressions required to establish a prototypical OCR algorithm to detect such changes. Efforts are underway to further improve the OCR algorithm by adding facial touching and automating application of the deformable masks and OCR algorithms to video footage of the moving faces as a prelude to blind validation of the automated approach.