Quantum-accessible reinforcement learning beyond strictly epochal environments.

In recent years, quantum-enhanced machine learning has emerged as a particularly fruitful application of quantum algorithms, covering aspects of supervised, unsupervised and reinforcement learning. Reinforcement learning offers numerous options of how quantum theory can be applied, and is arguably the least explored, from a quantum perspective. Here, an agent explores an environment and tries to find a behavior optimizing some figure of merit. Some of the first approaches investigated settings where this exploration can be sped-up, by considering quantum analogs of classical environments, which can then be queried in superposition. If the environments have a strict periodic structure in time (i.e. are strictly episodic), such environments can be effectively converted to conventional oracles encountered in quantum information. However, in general environments, we obtain scenarios that generalize standard oracle tasks. In this work, we consider one such generalization, where the environment is not strictly episodic, which is mapped to an oracle identification setting with a changing oracle. We analyze this case and show that standard amplitude-amplification techniques can, with minor modifications, still be applied to achieve quadratic speed-ups. In addition, we prove that an algorithm based on Grover iterations is optimal for oracle identification even if the oracle changes over time in a way that the "rewarded space" is monotonically increasing. This result constitutes one of the first generalizations of quantum-accessible reinforcement learning.

Experimental quantum speed-up in reinforcement learning agents.

As the field of artificial intelligence advances, the demand for algorithms that can learn quickly and efficiently increases. An important paradigm within artificial intelligence is reinforcement learning1, where decision-making entities called agents interact with environments and learn by updating their behaviour on the basis of the obtained feedback. The crucial question for practical applications is how fast agents learn2. Although various studies have made use of quantum mechanics to speed up the agent's decision-making process3,4, a reduction in learning time has not yet been demonstrated. Here we present a reinforcement learning experiment in which the learning process of an agent is sped up by using a quantum communication channel with the environment. We further show that combining this scenario with classical communication enables the evaluation of this improvement and allows optimal control of the learning progress. We implement this learning protocol on a compact and fully tunable integrated nanophotonic processor. The device interfaces with telecommunication-wavelength photons and features a fast active-feedback mechanism, demonstrating the agent's systematic quantum advantage in a setup that could readily be integrated within future large-scale quantum communication networks.

Creation and Characterization of Matter-Wave Breathers.

We report the creation of quasi-1D excited matter-wave solitons, "breathers," by quenching the strength of the interactions in a Bose-Einstein condensate with attractive interactions. We characterize the resulting breathing dynamics and quantify the effects of the aspect ratio of the confining potential, the strength of the quench, and the proximity of the 1D-3D crossover for the two-soliton breather. Furthermore, we demonstrate the complex dynamics of a three-soliton breather created by a stronger interaction quench. Our experimental results, which compare well with numerical simulations, provide a pathway for utilizing matter-wave breathers to explore quantum effects in large many-body systems.

On the convergence of projective-simulation-based reinforcement learning in Markov decision processes.

In recent years, the interest in leveraging quantum effects for enhancing machine learning tasks has significantly increased. Many algorithms speeding up supervised and unsupervised learning were established. The first framework in which ways to exploit quantum resources specifically for the broader context of reinforcement learning were found is projective simulation. Projective simulation presents an agent-based reinforcement learning approach designed in a manner which may support quantum walk-based speedups. Although classical variants of projective simulation have been benchmarked against common reinforcement learning algorithms, very few formal theoretical analyses have been provided for its performance in standard learning scenarios. In this paper, we provide a detailed formal discussion of the properties of this model. Specifically, we prove that one version of the projective simulation model, understood as a reinforcement learning approach, converges to optimal behavior in a large class of Markov decision processes. This proof shows that a physically inspired approach to reinforcement learning can guarantee to converge.

Long-Range Big Quantum-Data Transmission.

We introduce an alternative type of quantum repeater for long-range quantum communication with improved scaling with the distance. We show that by employing hashing, a deterministic entanglement distillation protocol with one-way communication, one obtains a scalable scheme that allows one to reach arbitrary distances, with constant overhead in resources per repeater station, and ultrahigh rates. In practical terms, we show that, also with moderate resources of a few hundred qubits at each repeater station, one can reach intercontinental distances. At the same time, a measurement-based implementation allows one to tolerate high loss but also operational and memory errors of the order of several percent per qubit. This opens the way for long-distance communication of big quantum data.

Monopole excitations of a harmonically trapped one-dimensional Bose gas from the ideal gas to the Tonks-Girardeau regime.

Using a time-dependent modified nonlinear Schrödinger equation (MNLSE)-where the conventional chemical potential proportional to the density is replaced by the one inferred from Lieb-Liniger's exact solution-we study frequencies of the collective monopole excitations of a one-dimensional Bose gas. We find that our method accurately reproduces the results of a recent experimental study [E. Haller et al., Science 325, 1224 (2009)] in the full spectrum of interaction regimes from the ideal gas, through the mean-field regime, through the mean-field Thomas-Fermi regime, all the way to the Tonks-Giradeau gas. While the former two are accessible by the standard time-dependent NLSE and inaccessible by the time-dependent local density approximation, the situation reverses in the latter case. However, the MNLSE is shown to treat all these regimes within a single numerical method.

Bosons in cigar-shaped traps: Thomas-Fermi regime, Tonks-Girardeau regime, and in between.

We present a quantitative analysis of the experimental accessibility of the Tonks-Girardeau gas in present-day experiments with cigar-trapped alkalis. For this purpose we derive, using a Bethe ansatz generated local equation of state, a set of hydrostatic equations describing one-dimensional, delta-interacting Bose gases trapped in a harmonic potential. The resulting solutions cover the entire range of atomic densities.