A Perceptual Measure for Deep Single Image Camera and Lens Calibration.

Image editing and compositing have become ubiquitous in entertainment, from digital art to AR and VR experiences. To produce beautiful composites, the camera needs to be geometrically calibrated, which can be tedious and requires a physical calibration target. In place of the traditional multi-image calibration process, we propose to infer the camera calibration parameters such as pitch, roll, field of view, and lens distortion directly from a single image using a deep convolutional neural network. We train this network using automatically generated samples from a large-scale panorama dataset, yielding competitive accuracy in terms of standard l2 error. However, we argue that minimizing such standard error metrics might not be optimal for many applications. In this work, we investigate human sensitivity to inaccuracies in geometric camera calibration. To this end, we conduct a large-scale human perception study where we ask participants to judge the realism of 3D objects composited with correct and biased camera calibration parameters. Based on this study, we develop a new perceptual measure for camera calibration and demonstrate that our deep calibration network outperforms previous single-image based calibration methods both on standard metrics as well as on this novel perceptual measure. Finally, we demonstrate the use of our calibration network for several applications, including virtual object insertion, image retrieval, and compositing.

Reducing touching eyes, nose and mouth ('T-zone') to reduce the spread of infectious disease: A prospective study of motivational, volitional and non-reflective predictors.

BACKGROUND: The route into the body for many pathogens is through the eyes, nose and mouth (i.e., the 'T-zone') via inhalation or fomite-based transfer during face touching. It is important to understand factors that are associated with touching the T-zone to inform preventive strategies. PURPOSE: To identify theory-informed predictors of intention to reduce facial 'T-zone' touching and self-reported 'T-zone' touching. METHODS: We conducted a nationally representative prospective questionnaire study of Canadians. Respondents were randomized to answer questions about touching their eyes, nose, or mouth with a questionnaire assessing 11 factors from an augmented Health Action Process Approach at baseline: intention, outcome expectancies, risk perception, individual severity, self-efficacy, action planning, coping planning, social support, automaticity, goal facilitation and stability of context. At 2-week follow-up, we assessed HAPA-based indicators of self-regulatory activities (awareness of standards, effort, self-monitoring) and self-reported behaviour (primary dependent variable). RESULTS: Of 656 Canadian adults recruited, 569 responded to follow-up (87% response rate). Across all areas of the 'T-zone', outcome expectancy was the strongest predictor of intention to reduce facial 'T-zone' touching, while self-efficacy was a significant predictor for only the eyes and mouth. Automaticity was the strongest predictor of behaviour at the 2-week follow-up. No sociodemographic or psychological factors predicted behaviour, with the exception of self-efficacy, which negatively predicted eye touching. CONCLUSION: Findings suggest that focusing on reflective processes may increase intention to reduce 'T-zone' touching, while reducing actual 'T-zone' touching may require strategies that address the automatic nature of this behaviour.

Machine learning techniques to characterize functional traits of plankton from image data.

Plankton imaging systems supported by automated classification and analysis have improved ecologists' ability to observe aquatic ecosystems. Today, we are on the cusp of reliably tracking plankton populations with a suite of lab-based and in situ tools, collecting imaging data at unprecedentedly fine spatial and temporal scales. But these data have potential well beyond examining the abundances of different taxa; the individual images themselves contain a wealth of information on functional traits. Here, we outline traits that could be measured from image data, suggest machine learning and computer vision approaches to extract functional trait information from the images, and discuss promising avenues for novel studies. The approaches we discuss are data agnostic and are broadly applicable to imagery of other aquatic or terrestrial organisms.

Imagery datasets for photobiological lighting analysis of architectural models with shading panels.

This paper describes eight imagery datasets including around 12000 images grouped in 1220 sets. The images were captured inside an architectural model aimed at exploring the impact of shading panels on photobiological lighting parameters. The architectural model represents a generic space at 1:10 scale with a single side fully glazing façade used to install shading panels. The datasets present interior lighting conditions under different shading configurations in terms of surface colors and glossiness, horizontal and vertical orientations and upwards, downwards, and left/right inclinations of panels, V-shape opening, low to high densities, and top and bottom positions at the window. The experiments of shading panel configurations were conducted under four to six different exterior overcast daylighting conditions simulated with very cool to very warm color temperatures and high to low intensities inside an artificial sky chamber. The datasets include bracketed low dynamic range (LDR) images which enable generating high dynamic range (HDR) images for photobiological lighting evaluations. Images were captured from the side and back viewpoints inside the model by using Raspberry Pi camera modules mounted with fisheye lenses. The datasets are reusable and useful for architects, lighting designers, and building engineers to study the impact of architectural variables and shading panels on photobiological lighting conditions in space. The datasets will also be interesting for computer vision specialists to run machine learning techniques and train artificial intelligence for architectural applications. The datasets are partially used in Parsaee, et al. [1]. The datasets are compiled as part of a doctoral dissertation in architecture at Laval University authored by Mojtaba Parsaee [2]. The datasets are shared through two Mendeley data repositories [3,4].

Inferring the solution space of microscope objective lenses using deep learning.

Lens design extrapolation (LDE) is a data-driven approach to optical design that aims to generate new optical systems inspired by reference designs. Here, we build on a deep learning-enabled LDE framework with the aim of generating a significant variety of microscope objective lenses (MOLs) that are similar in structure to the reference MOLs, but with varied sequences-defined as a particular arrangement of glass elements, air gaps, and aperture stop placement. We first formulate LDE as a one-to-many problem-specifically, generating varied lenses for any set of specifications and lens sequence. Next, by quantifying the structure of a MOL from the slopes of its marginal ray, we improve the training objective to capture the structures of the reference MOLs (e.g., Double-Gauss, Lister, retrofocus, etc.). From only 34 reference MOLs, we generate designs across 7432 lens sequences and show that the inferred designs accurately capture the structural diversity and performance of the dataset. Our contribution answers two current challenges of the LDE framework: incorporating a meaningful one-to-many mapping, and successfully extrapolating to lens sequences unseen in the dataset-a problem much harder than the one of extrapolating to new specifications.

Deep learning-enabled framework for automatic lens design starting point generation.

We present a simple, highly modular deep neural network (DNN) framework to address the problem of automatically inferring lens design starting points tailored to the desired specifications. In contrast to previous work, our model can handle various and complex lens structures suitable for real-world problems such as Cooke Triplets or Double Gauss lenses. Our successfully trained dynamic model can infer lens designs with realistic glass materials whose optical performance compares favorably to reference designs from the literature on 80 different lens structures. Using our trained model as a backbone, we make available to the community a web application that outputs a selection of varied, high-quality starting points directly from the desired specifications, which we believe will complement any lens designer's toolbox.

Single Day Outdoor Photometric Stereo.

Photometric Stereo (PS) under outdoor illumination remains a challenging, ill-posed problem due to insufficient variability in illumination. Months-long capture sessions are typically used in this setup, with little success on shorter, single-day time intervals. In this paper, we investigate the solution of outdoor PS over a single day, under different weather conditions. First, we investigate the relationship between weather and surface reconstructability in order to understand when natural lighting allows existing PS algorithms to work. Our analysis reveals that partially cloudy days improve the conditioning of the outdoor PS problem while sunny days do not allow the unambiguous recovery of surface normals from photometric cues alone. We demonstrate that calibrated PS algorithms can thus be employed to reconstruct Lambertian surfaces accurately under partially cloudy days. Second, we solve the ambiguity arising in clear days by combining photometric cues with prior knowledge on material properties, local surface geometry and the natural variations in outdoor lighting through a CNN-based, weakly-calibrated PS technique. Given a sequence of outdoor images captured during a single sunny day, our method robustly estimates the scene surface normals with unprecedented quality for the considered scenario. Our approach does not require precise geolocation and significantly outperforms several state-of-the-art methods on images with real lighting, showing that our CNN can combine efficiently learned priors and photometric cues available during a single sunny day.

Extrapolating from lens design databases using deep learning.

We propose for the first time a deep learning approach in assisting lens designers to find a lens design starting point. Using machine learning, lens design databases can be expanded in a continuous way to produce high-quality starting points from various optical specifications. A deep neural network (DNN) is trained to reproduce known forms of design (supervised training) and to jointly optimize the optical performance (unsupervised training) for generalization. In this work, the DNN infers high-performance cemented and air-spaced doublets that are tailored to diverse desired specifications after being fed with reference designs from the literature. The framework can be extended to lens systems with more optical surfaces.

Deep 6-DOF Tracking.

We present a temporal 6-DOF tracking method which leverages deep learning to achieve state-of-the-art performance on challenging datasets of real world capture. Our method is both more accurate and more robust to occlusions than the existing best performing approaches while maintaining real-time performance. To assess its efficacy, we evaluate our approach on several challenging RGBD sequences of real objects in a variety of conditions. Notably, we systematically evaluate robustness to occlusions through a series of sequences where the object to be tracked is increasingly occluded. Finally, our approach is purely data-driven and does not require any hand-designed features: robust tracking is automatically learned from data.

Engineered calcium-precipitable restriction enzyme.

We have developed a simple system for tagging and purifying proteins. Recent experiments have demonstrated that RTX (Repeat in Toxin) motifs from the adenylate cyclase toxin gene (CyaA) of B. pertussis undergo a conformational change upon binding calcium, resulting in precipitation of fused proteins and making this method a viable alternative for bioseparation. We have designed an iGEM Biobrick comprised of an RTX tag that can be easily fused to any protein of interest. In this paper, we detail the process of creating an RTX tagged version of the restriction enzyme EcoRI and describe a method for expression and purification of the functional enzyme.