Certifiable Robustness to Adversarial State Uncertainty in Deep Reinforcement Learning.

Deep neural network-based systems are now state-of-the-art in many robotics tasks, but their application in safety-critical domains remains dangerous without formal guarantees on network robustness. Small perturbations to sensor inputs (from noise or adversarial examples) are often enough to change network-based decisions, which was recently shown to cause an autonomous vehicle to swerve into another lane. In light of these dangers, numerous algorithms have been developed as defensive mechanisms from these adversarial inputs, some of which provide formal robustness guarantees or certificates. This work leverages research on certified adversarial robustness to develop an online certifiably robust for deep reinforcement learning algorithms. The proposed defense computes guaranteed lower bounds on state-action values during execution to identify and choose a robust action under a worst case deviation in input space due to possible adversaries or noise. Moreover, the resulting policy comes with a certificate of solution quality, even though the true state and optimal action are unknown to the certifier due to the perturbations. The approach is demonstrated on a deep Q-network (DQN) policy and is shown to increase robustness to noise and adversaries in pedestrian collision avoidance scenarios, a classic control task, and Atari Pong. This article extends our prior work with new performance guarantees, extensions to other reinforcement learning algorithms, expanded results aggregated across more scenarios, an extension into scenarios with adversarial behavior, comparisons with a more computationally expensive method, and visualizations that provide intuition about the robustness algorithm.

Distributed Certifiably Correct Pose-Graph Optimization.

This paper presents the first certifiably correct algorithm for distributed pose-graph optimization (PGO), the backbone of modern collaborative simultaneous localization and mapping (CSLAM) and camera network localization (CNL) systems. Our method is based upon a sparse semidefinite relaxation that we prove provides globally-optimal PGO solutions under moderate measurement noise (matching the guarantees enjoyed by state-of-the-art centralized methods), but is amenable to distributed optimization using the low-rank Riemannian Staircase framework. To implement the Riemannian Staircase in the distributed setting, we develop Riemannian block coordinate descent (RBCD), a novel method for (locally) minimizing a function over a product of Riemannian manifolds. We also propose the first distributed solution verification and saddle escape methods to certify the global optimality of critical points recovered via RBCD, and to descend from suboptimal critical points (if necessary). All components of our approach are inherently decentralized: they require only local communication, provide privacy protection, and are easily parallelizable. Extensive evaluations on synthetic and real-world datasets demonstrate that the proposed method correctly recovers globally optimal solutions under moderate noise, and outperforms alternative distributed techniques in terms of solution precision and convergence speed.

Vision-Based Multirotor Following Using Synthetic Learning Techniques.

Deep- and reinforcement-learning techniques have increasingly required large sets of real data to achieve stable convergence and generalization, in the context of image-recognition, object-detection or motion-control strategies. On this subject, the research community lacks robust approaches to overcome unavailable real-world extensive data by means of realistic synthetic-information and domain-adaptation techniques. In this work, synthetic-learning strategies have been used for the vision-based autonomous following of a noncooperative multirotor. The complete maneuver was learned with synthetic images and high-dimensional low-level continuous robot states, with deep- and reinforcement-learning techniques for object detection and motion control, respectively. A novel motion-control strategy for object following is introduced where the camera gimbal movement is coupled with the multirotor motion during the multirotor following. Results confirm that our present framework can be used to deploy a vision-based task in real flight using synthetic data. It was extensively validated in both simulated and real-flight scenarios, providing proper results (following a multirotor up to 1.3 m/s in simulation and 0.3 m/s in real flights).

Modeling and Planning with Macro-Actions in Decentralized POMDPs.

Decentralized partially observable Markov decision processes (Dec-POMDPs) are general models for decentralized multi-agent decision making under uncertainty. However, they typically model a problem at a low level of granularity, where each agent's actions are primitive operations lasting exactly one time step. We address the case where each agent has macro-actions: temporally extended actions that may require different amounts of time to execute. We model macro-actions as options in a Dec-POMDP, focusing on actions that depend only on information directly available to the agent during execution. Therefore, we model systems where coordination decisions only occur at the level of deciding which macro-actions to execute. The core technical difficulty in this setting is that the options chosen by each agent no longer terminate at the same time. We extend three leading Dec-POMDP algorithms for policy generation to the macro-action case, and demonstrate their effectiveness in both standard benchmarks and a multi-robot coordination problem. The results show that our new algorithms retain agent coordination while allowing high-quality solutions to be generated for significantly longer horizons and larger state-spaces than previous Dec-POMDP methods. Furthermore, in the multi-robot domain, we show that, in contrast to most existing methods that are specialized to a particular problem class, our approach can synthesize control policies that exploit opportunities for coordination while balancing uncertainty, sensor information, and information about other agents.

Dynamic Clustering Algorithms via Small-Variance Analysis of Markov Chain Mixture Models.

Bayesian nonparametrics are a class of probabilistic models in which the model size is inferred from data. A recently developed methodology in this field is small-variance asymptotic analysis, a mathematical technique for deriving learning algorithms that capture much of the flexibility of Bayesian nonparametric inference algorithms, but are simpler to implement and less computationally expensive. Past work on small-variance analysis of Bayesian nonparametric inference algorithms has exclusively considered batch models trained on a single, static dataset, which are incapable of capturing time evolution in the latent structure of the data. This work presents a small-variance analysis of the maximum a posteriori filtering problem for a temporally varying mixture model with a Markov dependence structure, which captures temporally evolving clusters within a dataset. Two clustering algorithms result from the analysis: D-Means, an iterative clustering algorithm for linearly separable, spherical clusters; and SD-Means, a spectral clustering algorithm derived from a kernelized, relaxed version of the clustering problem. Empirical results from experiments demonstrate the advantages of using D-Means and SD-Means over contemporary clustering algorithms, in terms of both computational cost and clustering accuracy.

Online Regression for Data With Changepoints Using Gaussian Processes and Reusable Models.

Many prediction, decision-making, and control architectures rely on online learned Gaussian process (GP) models. However, most existing GP regression algorithms assume a single generative model, leading to poor predictive performance when the data are nonstationary, i.e., generated from multiple switching processes. Furthermore, existing methods for GP regression over nonstationary data require significant computation, do not come with provable guarantees on correctness and speed, and many only work in batch settings, making them ill-suited for real-time prediction. We present an efficient online GP framework, GP-non-Bayesian clustering (GP-NBC), which addresses these computational and theoretical issues, allowing for real-time changepoint detection and regression using GPs. Our empirical results on two real-world data sets and two synthetic data set show that GP-NBC outperforms state-of-the-art methods for nonstationary regression in terms of both regression error and computation. For example, it outperforms Dirichlet process GP clustering with Gibbs sampling by 98% in computation time reduction while the mean absolute error is comparable.

Factors affecting discharge destination following lung transplantation.

BACKGROUND: Lung transplant (LT) recipients requiring additional care may be referred to inpatient rehabilitation prior to discharge home. This study seeks to describe discharge destinations following LT, compare the characteristics of patients discharged to different destinations, and identify the predictors of discharge destination. METHODS: Retrospective study of 243 LT recipients who survived to hospital discharge between 2006 and 2009. LT recipients were compared based on discharge destination on data pertaining to demographics, clinical characteristics, and healthcare utilization. RESULTS: Of the 243 recipients, 197 (81%) were discharged home, 42 (17%) to inpatient rehabilitation, and 4 (2%) to other medical facilities. Age, pulmonary diagnosis, most recent six-minute walk distance (6 MWD) prior to transplant, pre-transplant mechanical ventilation, priority listing status, pre- and post-transplant intensive care unit length of stay (ICU LOS), post-transplant LOS, total LOS, and participation in pre-transplant rehabilitation were statistically different between patients that were discharged home versus inpatient rehabilitation. Age, most recent 6 MWD prior to transplant, pre-transplant mechanical ventilation, and total LOS were found to be independent predictors of discharge destination. CONCLUSION: Clinical factors can help identify patients more likely to require inpatient rehabilitation. Identification of these factors has the potential to facilitate early discharge planning and optimize continuity of care.

A phase I trial of two sequence-specific schedules of decitabine and vorinostat in patients with acute myeloid leukemia.

This phase I trial evaluated two schedules of escalating vorinostat in combination with decitabine every 28 days: (i) sequential or (ii) concurrent. There were three dose-limiting toxicities: grade 3 fatigue and generalized muscle weakness on the sequential schedule (n = 1) and grade 3 fatigue on the concurrent schedule (n = 2). The maximum tolerated dose was not reached on both planned schedules. The overall response rate (ORR) was 23% (three complete response [CR], two CR with incomplete incomplete blood count recovery [CRi], one partial response [PR] and two morphological leukemic free state [MLFS]). The ORR for all and previously untreated patients in the sequential arm was 13% (one CRi; one MLFS) and 0% compared to 30% (three CR; one CRi; one PR; one MLFS) and 36% in the concurrent arm (p = 0.26 for both), respectively. Decitabine plus vorinostat was safe and has clinical activity in patients with previously untreated acute myeloid leukemia. Responses appear higher with the concurrent dose schedule. Cumulative toxicities may limit long-term usage on the current dose/schedules.

Bayesian nonparametric adaptive control using Gaussian processes.

Most current model reference adaptive control (MRAC) methods rely on parametric adaptive elements, in which the number of parameters of the adaptive element are fixed a priori, often through expert judgment. An example of such an adaptive element is radial basis function networks (RBFNs), with RBF centers preallocated based on the expected operating domain. If the system operates outside of the expected operating domain, this adaptive element can become noneffective in capturing and canceling the uncertainty, thus rendering the adaptive controller only semiglobal in nature. This paper investigates a Gaussian process-based Bayesian MRAC architecture (GP-MRAC), which leverages the power and flexibility of GP Bayesian nonparametric models of uncertainty. The GP-MRAC does not require the centers to be preallocated, can inherently handle measurement noise, and enables MRAC to handle a broader set of uncertainties, including those that are defined as distributions over functions. We use stochastic stability arguments to show that GP-MRAC guarantees good closed-loop performance with no prior domain knowledge of the uncertainty. Online implementable GP inference methods are compared in numerical simulations against RBFN-MRAC with preallocated centers and are shown to provide better tracking and improved long-term learning.

Hepatic ALECT-2 amyloidosis causing portal hypertension and recurrent variceal bleeding: a case report and review of the literature.

OBJECTIVES: To examine laser microdissection and mass spectrometry (LMD-MS), which has emerged as a new tool to aid in typing amyloid proteins. RESULTS: ALECT-2 is a potential cause of hepatic amyloidosis best detected by LMD-MS. METHODS: One of the more recently reported proteins is ALECT-2 (leukocyte cell-derived chemotaxin 2) amyloid, found in renal specimens of Hispanic patients. Here we report the first case of hepatic ALECT-2 amyloidosis diagnosed by LMD-MS from a liver biopsy specimen of a 52-year-old Hispanic man and causing portal hypertension with recurrent esophageal variceal bleeding. CONCLUSIONS: ALECT-2 can cause amyloidosis in organs other than the kidneys. It should be strongly considered in Hispanic patients and in those with a globular pattern of amyloid deposition. The incidence of ALECT-2 amyloidosis is likely underreported.

Comprehensive analysis and identification of the human STIM1 domains for structural and functional studies.

STIM1 is a Ca(2+) sensor within the ER membrane known to activate the plasma membrane store-operated Ca(2+) channel upon depletion of its target ion in the ER lumen. This activation is a crucial step to initiate the Ca(2+) signaling cascades within various cell types. Human STIM1 is a 77.4 kDa protein consisting of various domains that are involved in Ca(2+) sensing, oligomerization, and channel activation and deactivation. In this study, we identify the domains and boundaries in which functional and stable recombinant human STIM1 can be produced in large quantities. To achieve this goal, we cloned nearly 200 constructs that vary in their initial and terminal residues, length and presence of the transmembrane domain, and we conducted expression and purification analyses using these constructs. The results revealed that nearly half of the constructs could be expressed and purified with high quality, out of which 25% contained the integral membrane domain. Further analyses using surface plasmon resonance, nuclear magnetic resonance and a thermostability assay verified the functionality and integrity of these constructs. Thus, we have been able to identify the most stable and well-behaved domains of the hSTIM1 protein, which can be used for future in vitro biochemical and biophysical studies.

Influence of FLT3-internal tandem duplication allele burden and white blood cell count on the outcome in patients with intermediate-risk karyotype acute myeloid leukemia.

BACKGROUND: In patients with acute myeloid leukemia (AML), testing for fms-like tyrosine kinase-3 (FLT3)-internal tandem duplication (FLT3-ITD) and nucleophosmin-1 (NPM1) mutations can allow for further prognostic subclassification, but less is known about the effects of FLT3-ITD allele burden and presenting white blood cell count (WBC) within molecular subgroups. METHODS: The authors retrospectively assessed 206 adult patients who had AML with an intermediate-risk karyotype and who received treatment on a uniform induction and consolidation chemotherapy regimen. RESULTS: The presenting WBC was a prognostic factor for survival only in patients who had an FLT3-ITD mutation. On multivariate analysis, after correcting for age, WBC, secondary AML, and blast percentage, nucleophosmin-1 (NPM1)-mutated/FLT3-ITD-negative patients had superior overall survival compared with patients in the other molecular subgroups. Patients who had FLT3-ITD mutations had an inferior overall survival compared with patients who had NPM1 wild-type/FLT3-negative disease, and patients who had low or intermediate levels of the FLT-ITD of mutant allele had overall and disease-free survival similar to those in patients who had high-level mutations. CONCLUSIONS: NPM1 and FLT3-ITD status, age, WBC, and secondary AML were identified as important prognostic variables that can help to risk stratify patients with AML who have intermediate-risk cytogenetics. FLT3 allele burden had no significant influence on outcomes after correcting for other variables.

ENMD-2076 for hematological malignancies.

INTRODUCTION: Aurora kinases are key regulators of mitosis and inhibition of Aurora kinase activity is a rational therapeutic strategy in the treatment of solid tumors and hematological malignancies. AREAS COVERED: This paper will provide an updated summary of preclinical and clinical experience with ENMD-2076 in hematological malignancies. The MEDLINE (OVID) (1980 through 31 January 2012) was searched with the term combinations including Aurora, multiple myeloma, leukemia, lymphoma, myelodysplastic syndrome and myeloproliferative neoplasms. In addition, the American Society of Clinical Oncology (ASCO) (1997 - 2011) and the American Society of Hematology (ASH) (1997 - 2011) conference proceedings were searched for reports of new or ongoing trials. EXPERT OPINION: ENMD-2076 is a multi-kinase inhibitor, with activity against Aurora A kinase, FLT3, c-KIT, c-FMS and VEGFR-2 and -3. It appears to be tolerable, exhibits favorable pharmacokinetic profiles and has activity in patients with acute myeloid leukemia and multiple myeloma. Further evaluation with cytotoxic chemotherapy and targeted agents, which affect different pathways and have non-overlapping toxicities, in patients with hematological malignancies are warranted.

Autonomous driving in urban environments: approaches, lessons and challenges.

The development of autonomous vehicles for urban driving has seen rapid progress in the past 30 years. This paper provides a summary of the current state of the art in autonomous driving in urban environments, based primarily on the experiences of the authors in the 2007 DARPA Urban Challenge (DUC). The paper briefly summarizes the approaches that different teams used in the DUC, with the goal of describing some of the challenges that the teams faced in driving in urban environments. The paper also highlights the long-term research challenges that must be overcome in order to enable autonomous driving and points to opportunities for new technologies to be applied in improving vehicle safety, exploiting intelligent road infrastructure and enabling robotic vehicles operating in human environments.

Beta-amyloid increases somatostatin expression in cultured cortical neurons.

In beta-amyloid (Abeta)-induced neurotoxicity, activation of the NMDA receptor, increased Ca2+ and oxidative stress are intimately associated with neuronal cell death as normally seen in NMDA-induced neurotoxicity. We have recently shown selective sparing of somatostatin (SST)-positive neurons and increased SST expression in NMDA agonist-induced neurotoxicity. Accordingly, the present study was undertaken to determine the effect of Abeta25-35-induced neurotoxicity on the expression of SST in cultured cortical neurons. Cultured cortical cells were exposed to Abeta25-35 and processed to determine the cellular content and release of SST into medium by radioimmunoassay and SST mRNA by RT-PCR. Abeta25-35 induces neuronal cell death in a concentration- and time-dependent fashion, increases SST mRNA synthesis and induces an augmentation in the cellular content of SST. No significant changes were seen on SST release at any concentration of Abeta25-35 after 24 h of treatment. However, Abeta25-35 induces a significant increase of SST release into medium only after 12 h in comparison with other time points. Most significantly, SST-positive neurons are selectively spared in the presence of a lower concentration of Abeta25-35, whereas, in the presence of higher concentrations of Abeta25-35 for extended time periods, SST-positive neurons decrease gradually. Furthermore, Abeta25-35 induces apoptosis at lower concentrations (5 and 10 micromol/L) and necrosis at higher concentrations (20 and 40 micromol/L). Consistent with the increased accumulation of SST, these data suggest that Abeta25-35 impairs cell membrane permeability. Selective sparing of SST-positive neurons at lower concentrations of Abeta25-35 at early time points directly correlates with the pathophysiology of Alzheimer's disease.