Free-viewpoint video of human actors using multiple handheld Kinects.

We present an algorithm for creating free-viewpoint video of interacting humans using three handheld Kinect cameras. Our method reconstructs deforming surface geometry and temporal varying texture of humans through estimation of human poses and camera poses for every time step of the RGBZ video. Skeletal configurations and camera poses are found by solving a joint energy minimization problem, which optimizes the alignment of RGBZ data from all cameras, as well as the alignment of human shape templates to the Kinect data. The energy function is based on a combination of geometric correspondence finding, implicit scene segmentation, and correspondence finding using image features. Finally, texture recovery is achieved through jointly optimization on spatio-temporal RGB data using matrix completion. As opposed to previous methods, our algorithm succeeds on free-viewpoint video of human actors under general uncontrolled indoor scenes with potentially dynamic background, and it succeeds even if the cameras are moving.

Automatic track recognition of footprints for identifying cryptic species.

The recognition of tracks plays an important role in ecological research and monitoring, and tracking tunnels are a cost-effective method for indexing species over large areas. Traditionally, tracks are collected by a tracking system, and analysis is cairied out in a manual identification procedure by experienced wildlife biologists. Unfortunately, human experts are unable to reliably distinguish tracks of morphologically similar species. We propose a new method using image analysis, which allows automatic species identification of tracks, and apply the method to identifying cryptic small-mammal species. We demonstrate the method by identifying footprints of three invasive rat species with similar morphology that co-occur in New Zealand, including detection of a recent invasion of a rat-free island. Automatic footprint recognition successfully identified the species of rat for >70% of footprints, and >83% of tracking cards. With appropriate changes to the image recognition, the method could be broadly applicable to any taxa that can be tracked. Identification of tracks to species level gives better estimates of species presence and composition in communities.