Efficient nonparametric belief propagation for pose estimation and manipulation of articulated objects.

Robots working in human environments often encounter a wide range of articulated objects, such as tools, cabinets, and other jointed objects. Such articulated objects can take an infinite number of possible poses, as a point in a potentially high-dimensional continuous space. A robot must perceive this continuous pose to manipulate the object to a desired pose. This problem of perception and manipulation of articulated objects remains a challenge due to its high dimensionality and multimodal uncertainty. Here, we describe a factored approach to estimate the poses of articulated objects using an efficient approach to nonparametric belief propagation. We consider inputs as geometrical models with articulation constraints and observed RGBD (red, green, blue, and depth) sensor data. The described framework produces object-part pose beliefs iteratively. The problem is formulated as a pairwise Markov random field (MRF), where each hidden node (continuous pose variable) is an observed object-part's pose and the edges denote the articulation constraints between the parts. We describe articulated pose estimation by a "pull" message passing algorithm for nonparametric belief propagation (PMPNBP) and evaluate its convergence properties over scenes with articulated objects. Robot experiments are provided to demonstrate the necessity of maintaining beliefs to perform goal-driven manipulation tasks.

Estimation of human core temperature from sequential heart rate observations.

Core temperature (CT) in combination with heart rate (HR) can be a good indicator of impending heat exhaustion for occupations involving exposure to heat, heavy workloads, and wearing protective clothing. However, continuously measuring CT in an ambulatory environment is difficult. To address this problem we developed a model to estimate the time course of CT using a series of HR measurements as a leading indicator using a Kalman filter. The model was trained using data from 17 volunteers engaged in a 24 h military field exercise (air temperatures 24-36 °C, and 42%-97% relative humidity and CTs ranging from 36.0-40.0 °C). Validation data from laboratory and field studies (N = 83) encompassing various combinations of temperature, hydration, clothing, and acclimation state were examined using the Bland-Altman limits of agreement (LoA) method. We found our model had an overall bias of -0.03 ± 0.32 °C and that 95% of all CT estimates fall within ±0.63 °C (>52 000 total observations). While the model for estimating CT is not a replacement for direct measurement of CT (literature comparisons of esophageal and rectal methods average LoAs of ±0.58 °C) our results suggest it is accurate enough to provide practical indication of thermal work strain for use in the work place.

Dynamical simulation priors for human motion tracking.

We propose a simulation-based dynamical motion prior for tracking human motion from video in presence of physical ground-person interactions. Most tracking approaches to date have focused on efficient inference algorithms and/or learning of prior kinematic motion models; however, few can explicitly account for the physical plausibility of recovered motion. Here, we aim to recover physically plausible motion of a single articulated human subject. Toward this end, we propose a full-body 3D physical simulation-based prior that explicitly incorporates a model of human dynamics into the Bayesian filtering framework. We consider the motion of the subject to be generated by a feedback "control loop" in which Newtonian physics approximates the rigid-body motion dynamics of the human and the environment through the application and integration of interaction forces, motor forces, and gravity. Interaction forces prevent physically impossible hypotheses, enable more appropriate reactions to the environment (e.g., ground contacts), and are produced from detected human-environment collisions. Motor forces actuate the body, ensure that proposed pose transitions are physically feasible, and are generated using a motion controller. For efficient inference in the resulting high-dimensional state space, we utilize an exemplar-based control strategy that reduces the effective search space of motor forces. As a result, we are able to recover physically plausible motion of human subjects from monocular and multiview video. We show, both quantitatively and qualitatively, that our approach performs favorably with respect to Bayesian filtering methods with standard motion priors.

Human thermoregulatory system state estimation using non-invasive physiological sensors.

Small teams of emergency workers/military can often find themselves engaged in critical, high exertion work conducted under challenging environmental conditions. These types of conditions present thermal work strain challenges which unmitigated can lead to collapse (heat exhaustion) or even death from heat stroke. Physiological measurement of these teams provides a mechanism that could be an effective tool in preventing thermal injury. While indices of thermal work strain have been proposed they suffer from ignoring thermoregulatory context and rely on measuring internal temperature (IT). Measurement of IT in free ranging ambulatory environments is problematic. In this paper we propose a physiology based Dynamic Bayesian Network (DBN) model that estimates internal temperature, heat production and heat transfer from observations of heart rate, accelerometry, and skin heat flux. We learn the model's conditional probability distributions from seven volunteers engaged in a 48 hour military field training exercise. We demonstrate that sum of our minute to minute heat production estimates correlate well with total daily energy expenditure (TDEE) measured using the doubly labeled water technique (r(2) = 0.73). We also demonstrate that the DBN is able to infer IT in new datasets to within ±0.5 °C over 85% of the time. Importantly, the additional thermoregulatory context allows critical high IT temperature to be estimated better than previous approaches. We conclude that the DBN approach shows promise in enabling practical real time thermal work strain monitoring applications from physiological monitoring systems that exist today.

Interactions between causal models, theories, and social cognitive development.

We propose a model of social cognitive development based not on a single modeling framework or the hypothesis that a single model accounts for children's developing social cognition. Rather, we advocate a Causal Model approach (cf. Waldmann, 1996), in which models of social cognitive development take the same position as theories of social cognitive development, in that they generate novel empirical hypotheses. We describe this approach and present three examples across various aspects of social cognitive development. Our first example focuses on children's understanding of pretense and involves only considering assumptions made by a computational framework. The second example focuses on children's learning from "testimony". It uses a modeling framework based on Markov random fields as a computational description of a set of empirical phenomena, and then tests a prediction of that description. The third example considers infants' generalization of action learned from imitation. Here, we use a modified version of the Rational Model of Categorization to explain children's inferences. Taken together, these examples suggest that research in social cognitive development can be assisted by considering how computational modeling can lead researchers towards testing novel hypotheses.

2D subspaces for sparse control of high-DOF robots.

We investigate the use of five dimension reduction and manifold learning techniques to estimate a 2D subspace of hand poses for the purpose of generating motion. Our aim is to uncover a 2D parameterization from optical motion capture data that allows for transformation sparse user input trajectories into desired hand movements. The use of shape descriptors for representing hand pose is additionally explored for dealing with occluded parts of the hand during data collection. We present early results from uncovering 2D parameterizations of power and precision grasps and their use to drive a physically simulated hand from 2D mouse input.