Parallel Online Temporal Difference Learning for Motor Control.

Temporal difference (TD) learning, a key concept in reinforcement learning, is a popular method for solving simulated control problems. However, in real systems, this method is often avoided in favor of policy search methods because of its long learning time. But policy search suffers from its own drawbacks, such as the necessity of informed policy parameterization and initialization. In this paper, we show that TD learning can work effectively in real robotic systems as well, using parallel model learning and planning. Using locally weighted linear regression and trajectory sampled planning with 14 concurrent threads, we can achieve a speedup of almost two orders of magnitude over regular TD control on simulated control benchmarks. For a real-world pendulum swing-up task and a two-link manipulator movement task, we report a speedup of 20× to 60× , with a real-time learning speed of less than half a minute. The results are competitive with state-of-the-art policy search.

Specific visualization of glioma cells in living low-grade tumor tissue.

BACKGROUND: The current therapy of malignant gliomas is based on surgical resection, radio-chemotherapy and chemotherapy. Recent retrospective case-series have highlighted the significance of the extent of resection as a prognostic factor predicting the course of the disease. Complete resection in low-grade gliomas that show no MRI-enhanced images are especially difficult. The aim in this study was to develop a robust, specific, new fluorescent probe for glioma cells that is easy to apply to live tumor biopsies and could identify tumor cells from normal brain cells at all levels of magnification. METHODOLOGY/PRINCIPAL FINDINGS: In this investigation we employed brightly fluorescent, photostable quantum dots (QDs) to specifically target epidermal growth factor receptor (EGFR) that is upregulated in many gliomas. Living glioma and normal cells or tissue biopsies were incubated with QDs coupled to EGF and/or monoclonal antibodies against EGFR for 30 minutes, washed and imaged. The data include results from cell-culture, animal model and ex vivo human tumor biopsies of both low-grade and high-grade gliomas and show high probe specificity. Tumor cells could be visualized from the macroscopic to single cell level with contrast ratios as high as 1000: 1 compared to normal brain tissue. CONCLUSIONS/SIGNIFICANCE: The ability of the targeted probes to clearly distinguish tumor cells in low-grade tumor biopsies, where no enhanced MRI image was obtained, demonstrates the great potential of the method. We propose that future application of specifically targeted fluorescent particles during surgery could allow intraoperative guidance for the removal of residual tumor cells from the resection cavity and thus increase patient survival.

Characterization of coupled ground state and excited state equilibria by fluorescence spectral deconvolution.

Fluorescence probes with multiparametric response based on the relative variation in the intensities of several emission bands are of great general utility. An accurate interpretation of the system requires the determination of the number, positions and intensities of the spectral components. We have developed a new algorithm for spectral deconvolution that is applicable to fluorescence probes exhibiting a two-state ground-state equilibrium and a two-state excited-state reaction. Three distinct fluorescence emission bands are resolved, with a distribution of intensities that is excitation-wavelength-dependent. The deconvolution of the spectrum into individual components is based on their representation as asymmetric Siano-Metzler log-normal functions. The application of the algorithm to the solvation response of a 3-hydroxychromone (3HC) derivative that exhibits an H-bonding-dependent excited-state intramolecular proton transfer (ESIPT) reaction allowed the separation of the spectral signatures characteristic of polarity and hydrogen bonding. This example demonstrates the ability of the method to characterize two potentially uncorrelated parameters characterizing dye environment and interactions.

A triple-emission fluorescent probe reveals distinctive amyloid fibrillar polymorphism of wild-type alpha-synuclein and its familial Parkinson's disease mutants.

Intracytoplasmic neuronal deposits containing amyloid fibrils of the 140-amino acid presynaptic protein alpha-synuclein (AS) are the hallmark of Parkinson's (PD) disease and related neurodegenerative disorders. Three point mutations (A53T, A30P, and E46K) are linked to early onset PD. Compared to the wild-type (WT) protein, the mutants aggregate faster in vitro, but their fibrillar products are quite similar. Using the extrinsic multiple-emission probe 4'-(diethylamino)-3-hydroxyflavone (FE), we demonstrate unique and distinct spectroscopic signatures for the amyloid fibrils formed by the WT and mutant AS, presumably indicative of subtle differences in supramolecular structure. The two well-separated emission bands of the FE probe originate from a proton transfer reaction in the excited state. The ratiometric response constitutes a sensitive, tunable reporter of microenvironmental properties such as polarity and hydrogen bonding. The very distinctive fluorescence spectra of the FE probe bound to the four AS variants reflect different tautomeric equilibria in the excited state and the existence of at least two different binding sites in the fibrils for the dye. Deconvolution of the two-dimensional excitation-emission spectra leads to estimations of different local dielectric constants and extents of hydration characteristic of the proteins. The sensitivity of such a simple external probe to conformational alterations induced by point mutations is unprecedented and provides new insight into key phenomena related to amyloid fibrils: plasticity, polymorphism, propagation of structural features, and structure-function relationships underlying toxicity.

Tumor-targeted quantum dots can help surgeons find tumor boundaries.

Despite surgical advances and recent progress in adjuvant therapies, the prognosis for patients with malignant brain tumors such as glioblastoma multiforme has remained poor, and the neurological deterioration suffered by most patients as a consequence of tumor progression is dramatic and severe. In addition, malignant brain tumors have >>95% recurrence close to the primary site of initial resection. Unfortunately, standard imaging techniques do not permit the intraoperative identification of individual or small clusters of residual tumor cells, precluding their selective removal while sparing the surrounding normal brain tissue. In this report, we show that quantum dots (QDs) coupled to epidermal growth factor (EGF) or anti-EGF receptor receptor (EGFR, Her1) specifically and sensitively label glial tumor cells in cell culture, glioma mouse models, and human brain-tumor biopsies. A clear demarcation between brain and tumor tissue at the macroscopic as well as the cellular level is provided by the fluorescence emission of the QDs.

Fluorescence recovery after photobleaching and photoconversion in multiple arbitrary regions of interest using a programmable array microscope.

Photomanipulation (photobleaching, photoactivation, or photoconversion) is an essential tool in fluorescence microscopy. Fluorescence recovery after photobleaching (FRAP) is commonly used for the determination of lateral diffusion constants of membrane proteins, and can be conveniently implemented in confocal laser scanning microscopy (CLSM). Such determinations provide important information on molecular dynamics in live cells. However, the CLSM platform is inherently limited for FRAP because of its inflexible raster (spot) scanning format. We have implemented FRAP and photoactivation protocols using structured illumination and detection in a programmable array microscope (PAM). The patterns are arbitrary in number and shape, dynamic and adjustable to and by the sample characteristics. We have used multispot PAM-FRAP to measure the lateral diffusion of the erbB3 (HER3) receptor tyrosine kinase labeled by fusion with mCitrine on untreated cells and after treatment with reagents that perturb the cytoskeleton or plasma membrane or activate coexpressed erbB1 (HER1, the EGF receptor EGFR). We also show the versatility of the PAM for photoactivation in arbitrary regions of interest, in cells expressing erbB3 fused with the photoconvertible fluorescent protein dronpa.