Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations.

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimization tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorizable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process, the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.

Using (1+1)D quantum cellular automata for exploring collective effects in large-scale quantum neural networks.

Central to the field of quantum machine learning is the design of quantum perceptrons and neural network architectures. A key question in this regard is the impact of quantum effects on the way such models process information. Here, we establish a connection between (1+1)D quantum cellular automata, which implement a discrete nonequilibrium quantum many-body dynamics through successive applications of local quantum gates, and quantum neural networks (QNNs), which process information by feeding it through perceptrons interconnecting adjacent layers. Exploiting this link, we construct a class of QNNs that are highly structured-aiding both interpretability and helping to avoid trainability issues in machine learning tasks-yet can be connected rigorously to continuous-time Lindblad dynamics. We further analyze the universal properties of an example case, identifying a change of critical behavior when quantum effects are varied, showing their potential impact on the collective dynamical behavior underlying information processing in large-scale QNNs.

Asynchronism and nonequilibrium phase transitions in (1+1)-dimensional quantum cellular automata.

Probabilistic cellular automata provide a simple framework for exploring classical nonequilibrium processes. Recently, quantum cellular automata have been proposed that rely on the propagation of a one-dimensional quantum state along a fictitious discrete time dimension via the sequential application of quantum gates. The resulting (1+1)-dimensional space-time structure makes these automata special cases of recurrent quantum neural networks which can implement broad classes of classical nonequilibrium processes. Here, we present a general prescription by which these models can be extended into genuinely quantum nonequilibrium models via the systematic inclusion of asynchronism. This is illustrated for the classical contact process, where the resulting model is closely linked to the quantum contact process (QCP), developed in the framework of open quantum systems. Studying the mean-field behavior of the model, we find evidence of an "asynchronism transition," i.e., a sudden qualitative change in the phase transition behavior once a certain degree of asynchronicity is surpassed, a phenomenon we link to observations in the QCP.

Quantum and Classical Temporal Correlations in (1+1)D Quantum Cellular Automata.

We employ (1+1)-dimensional quantum cellular automata to study the evolution of entanglement and coherence near criticality in quantum systems that display nonequilibrium steady-state phase transitions. This construction permits direct access to the entire space-time structure of the underlying nonequilibrium dynamics, and allows for the analysis of unconventional correlations, such as entanglement in the time direction between the "present" and the "past." We show how the uniquely quantum part of these correlations-the coherence-can be isolated and that, close to criticality, its dynamics displays a universal power-law behavior on approach to stationarity. Focusing on quantum generalizations of classical nonequilibrium systems: the Domany-Kinzel cellular automaton and the Bagnoli-Boccara-Rechtman model, we estimate the universal critical exponents for both the entanglement and coherence. As these models belong to the one-dimensional directed percolation universality class, the latter provides a key new critical exponent, one that is unique to quantum systems.

Nonequilibrium Phase Transitions in (1+1)-Dimensional Quantum Cellular Automata with Controllable Quantum Correlations.

Motivated by recent progress in the experimental development of quantum simulators based on Rydberg atoms, we introduce and investigate the dynamics of a class of (1+1)-dimensional quantum cellular automata. These nonequilibrium many-body models, which are quantum generalizations of the Domany-Kinzel cellular automaton, possess two key features: they display stationary behavior and nonequilibrium phase transitions despite being isolated systems. Moreover, they permit the controlled introduction of local quantum correlations, which allows for the impact of quantumness on the dynamics and phase transition to be assessed. We show that projected entangled pair state tensor networks permit a natural and efficient representation of the cellular automaton. Here, the degree of quantumness and complexity of the dynamics is reflected in the difficulty of contracting the tensor network.

Critical Behavior of the Quantum Contact Process in One Dimension.

The contact process is a paradigmatic classical stochastic system displaying critical behavior even in one dimension. It features a nonequilibrium phase transition into an absorbing state that has been widely investigated and shown to belong to the directed percolation universality class. When the same process is considered in a quantum setting, much less is known. So far, mainly semiclassical studies have been conducted and the nature of the transition in low dimensions is still a matter of debate. Also, from a numerical point of view, from which the system may look fairly simple-especially in one dimension-results are lacking. In particular, the presence of the absorbing state poses a substantial challenge, which appears to affect the reliability of algorithms targeting directly the steady state. Here we perform real-time numerical simulations of the open dynamics of the quantum contact process and shed light on the existence and on the nature of an absorbing state phase transition in one dimension. We find evidence for the transition being continuous and provide first estimates for the critical exponents. Beyond the conceptual interest, the simplicity of the quantum contact process makes it an ideal benchmark problem for scrutinizing numerical methods for open quantum nonequilibrium systems.