Stochastic quasi-gradient methods: variance reduction via Jacobian sketching.

We develop a new family of variance reduced stochastic gradient descent methods for minimizing the average of a very large number of smooth functions. Our method-JacSketch-is motivated by novel developments in randomized numerical linear algebra, and operates by maintaining a stochastic estimate of a Jacobian matrix composed of the gradients of individual functions. In each iteration, JacSketch efficiently updates the Jacobian matrix by first obtaining a random linear measurement of the true Jacobian through (cheap) sketching, and then projecting the previous estimate onto the solution space of a linear matrix equation whose solutions are consistent with the measurement. The Jacobian estimate is then used to compute a variance-reduced unbiased estimator of the gradient. Our strategy is analogous to the way quasi-Newton methods maintain an estimate of the Hessian, and hence our method can be seen as a stochastic quasi-gradient method. Our method can also be seen as stochastic gradient descent applied to a controlled stochastic optimization reformulation of the original problem, where the control comes from the Jacobian estimates. We prove that for smooth and strongly convex functions, JacSketch converges linearly with a meaningful rate dictated by a single convergence theorem which applies to general sketches. We also provide a refined convergence theorem which applies to a smaller class of sketches, featuring a novel proof technique based on a stochastic Lyapunov function. This enables us to obtain sharper complexity results for variants of JacSketch with importance sampling. By specializing our general approach to specific sketching strategies, JacSketch reduces to the celebrated stochastic average gradient (SAGA) method, and its several existing and many new minibatch, reduced memory, and importance sampling variants. Our rate for SAGA with importance sampling is the current best-known rate for this method, resolving a conjecture by Schmidt et al. (Proceedings of the eighteenth international conference on artificial intelligence and statistics, AISTATS 2015, San Diego, California, 2015). The rates we obtain for minibatch SAGA are also superior to existing rates and are sufficiently tight as to show a decrease in total complexity as the minibatch size increases. Moreover, we obtain the first minibatch SAGA method with importance sampling.

In vivo capture and label-free detection of early metastatic cells.

Breast cancer is a leading cause of death for women, with mortality resulting from metastasis. Metastases are often detected once tumour cells affect the function of solid organs, with a high disease burden limiting effective treatment. Here we report a method for the early detection of metastasis using an implanted scaffold to recruit and capture metastatic cells in vivo, which achieves high cell densities and reduces the tumour burden within solid organs 10-fold. Recruitment is associated with infiltration of immune cells, which include Gr1(hi)CD11b(+) cells. We identify metastatic cells in the scaffold through a label-free detection system using inverse spectroscopic optical coherence tomography, which identifies changes to nanoscale tissue architecture associated with the presence of tumour cells. For patients at risk of recurrence, scaffold implantation following completion of primary therapy has the potential to identify metastatic disease at the earliest stage, enabling initiation of therapy while the disease burden is low.

Functional characterization of embryonic stem cell-derived endothelial cells.

Endothelial cells (EC) derived from embryonic stem cells (ESC) require additional functional characterization before they are used as a cell therapy in order to enhance their potential for engraftment and proliferation. We explore several physiologically relevant functions of ESC-derived EC (ESC-EC), such as its capacity to produce nitric oxide (NO), regulate permeability, activate and express surface molecules for the recruitment of leukocytes in response to inflammatory stimuli, migrate and grow new blood vessels, lay down extracellular matrix, and take up low-density lipoproteins. We also examined the ESC-EC ability to upregulate NO in response to shear stress and downregulate NO in response to pro-inflammatory TNF-α activation. Functional responses of ESC-EC were compared with those of cultured mouse aortic ECs. The ESC-EC exhibit most aspects of functional endothelium, but interesting differences remain. The ESC-EC produced less NO on a per cell basis, but the same amount of NO if quantified based on the area of endothelial tissue. They also exhibit increased angiogenic sprouting and are more resistant to inflammatory signals. We further characterized the subphenotype of our ESC-EC and observed both venous and arterial markers on individual cells with a larger percentage of the cells exhibiting a venous phenotype. These data support the hypothesis that the developmental default pathway is toward a venous EC, and that refinement of methods for differentiation towards arterial EC is required to maintain a homogeneous population.