How Trustworthy are Performance Evaluations for Basic Vision Tasks?

This article examines performance evaluation criteria for basic vision tasks involving sets of objects namely, object detection, instance-level segmentation and multi-object tracking. The rankings of algorithms by a criterion can fluctuate with different choices of parameters, e.g. Intersection over Union (IoU) threshold, making their evaluations unreliable. More importantly, there is no means to verify whether we can trust the evaluations of a criterion. This work suggests a notion of trustworthiness for performance criteria, which requires (i) robustness to parameters for reliability, (ii) contextual meaningfulness in sanity tests, and (iii) consistency with mathematical requirements such as the metric properties. We observe that these requirements were overlooked by many widely-used criteria, and explore alternative criteria using metrics for sets of shapes. We also assess all these criteria based on the suggested requirements for trustworthiness.

A Bayesian Filter for Multi-View 3D Multi-Object Tracking With Occlusion Handling.

This paper proposes an online multi-camera multi-object tracker that only requires monocular detector training, independent of the multi-camera configurations, allowing seamless extension/deletion of cameras without retraining effort. The proposed algorithm has a linear complexity in the total number of detections across the cameras, and hence scales gracefully with the number of cameras. It operates in the 3D world frame, and provides 3D trajectory estimates of the objects. The key innovation is a high fidelity yet tractable 3D occlusion model, amenable to optimal Bayesian multi-view multi-object filtering, which seamlessly integrates, into a single Bayesian recursion, the sub-tasks of track management, state estimation, clutter rejection, and occlusion/misdetection handling. The proposed algorithm is evaluated on the latest WILDTRACKS dataset, and demonstrated to work in very crowded scenes on a new dataset.

Stem cell migration and mechanotransduction on linear stiffness gradient hydrogels.

The spatial presentation of mechanical information is a key parameter for cell behavior. We have developed a method of polymerization control in which the differential diffusion distance of unreacted cross-linker and monomer into a prepolymerized hydrogel sink results in a tunable stiffness gradient at the cell-matrix interface. This simple, low-cost, robust method was used to produce polyacrylamide hydrogels with stiffness gradients of 0.5, 1.7, 2.9, 4.5, 6.8, and 8.2 kPa/mm, spanning the in vivo physiological and pathological mechanical landscape. Importantly, three of these gradients were found to be nondurotactic for human adipose-derived stem cells (hASCs), allowing the presentation of a continuous range of stiffnesses in a single well without the confounding effect of differential cell migration. Using these nondurotactic gradient gels, stiffness-dependent hASC morphology, migration, and differentiation were studied. Finally, the mechanosensitive proteins YAP, Lamin A/C, Lamin B, MRTF-A, and MRTF-B were analyzed on these gradients, providing higher-resolution data on stiffness-dependent expression and localization.

Multi-Target Tracking With Time-Varying Clutter Rate and Detection Profile: Application to Time-Lapse Cell Microscopy Sequences.

Quantitative analysis of the dynamics of tiny cellular and sub-cellular structures, known as particles, in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, complex motion patterns and intricate interactions. In this paper, we propose a framework for tracking these structures based on the random finite set Bayesian filtering framework. We focus on challenging biological applications where image characteristics such as noise and background intensity change during the acquisition process. Under these conditions, detection methods usually fail to detect all particles and are often followed by missed detections and many spurious measurements with unknown and time-varying rates. To deal with this, we propose a bootstrap filter composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability of the targets, while the tracker estimates the state of the targets. Our results show that the proposed approach can outperform state-of-the-art particle trackers on both synthetic and real data in this regime.

A multiple model probability hypothesis density tracker for time-lapse cell microscopy sequences.

Quantitative analysis of the dynamics of tiny cellular and subcellular structures in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, maneuvering motion patterns and intricate interactions. The linear Gaussian jump Markov system probability hypothesis density (LGJMS-PHD) filter is a recent Bayesian tracking filter that is well-suited for this task. However, the existing recursion equations for this filter do not consider a state-dependent transition probability matrix. As required in many biological applications, we propose a new closed-form recursion that incorporates this assumption and introduce a general framework for particle tracking using the proposed filter. We apply our scheme to multi-target tracking in total internal reflection fluorescence microscopy (TIRFM) sequences and evaluate the performance of our filter against the existing LGJMS-PHD and IMM-JPDA filters.