However, the successful implementation of most quantum technologies relies heavily on the ability to manipulate the underlying quantum systems. This task is already challenging in classical dynamics, but quantum physics adds an extra layer of complexity. The issue arises from the difficulty in simulating quantum systems with many interacting qubits, as the memory requirements exceed the capabilities of even the best classical supercomputers. Physicists refer to this challenge as the "curse of dimensionality", rendering it infeasible to simulate the behavior of large quantum many-body systems using classical computers and devising optimal control strategies for them.

Addressing this problem, a recent research article published in Nature Machine Intelligence, by Friederike Metz (OIST and EPFL) and Marin Bukov (Sofia University and MPI-PKS), introduces an innovative approach. The authors applied deep reinforcement learning (RL), a subfield of machine learning, to design an artificial intelligent agent capable of effectively controlling quantum many-body systems. To overcome the curse of dimensionality, they employed tensor networks – mathematical structures that allow for an approximate representation of large quantum states on classical computers. Leveraging tensor networks, Metz and Bukov developed a novel deep learning architecture that empowers RL agents to process and interpret quantum many-body states seamlessly.

The trained RL agent demonstrated remarkable performance in preparing ordered ground states in the quantum Ising chain, a fundamental model for studying quantum magnetism. This new framework surpassed the limitations of standard neural-network-only architectures, enabling the control of significantly larger systems while retaining the benefits of deep learning algorithms, such as generalizability and trainable robustness to noise. Notably, the RL agent exhibited the ability to find universal controls in few-qubit systems, learn to optimally steer previously unseen many-qubit states, and adapt control protocols in real-time when faced with stochastic perturbations in quantum dynamics. Additionally, the authors propose a way to map their RL framework to a hybrid quantum-classical algorithm that can be executed on noisy intermediate-scale quantum devices.

This research has profound implications, paving the way for applying deep RL to efficiently control large quantum systems – a crucial requirement for advancing modern quantum technologies. With these techniques, researchers expect to explore novel quantum phases, design complex molecules, achieve unprecedented measurement precision, and build secure networks using quantum communication, among other groundbreaking applications.*

Principal investigator of the research study is Marin Bukov, PhD (Max Planck Institute for the Physics of Compplex Systems, Dresden and Department of Physics, Sofia University until 2022). The simulations were carried out by Dr. Friederike Metz (Okinawa Institute for Science and Technology, Japan and Swiss Federal Institute of Technology in Lausanne, Switzerland) as part of her PhD thesis. This research was funded by the VIHREN frontier research program of the Bulgarian National Science Fund (contract no KP-06-DV-5, until 25.06.2021), and the Marie Sklodowska-Curie Actions as part of European Commission’s Horizon 2020 program (grant number 890711).

* an earlier version of this text was improved using ChatGPT
** The image accompanying this text was created with the assistance of DALL·E 2. Prompt: A robot manipulating atoms in a quantum computer, Surrealism

contact: mgbukov(at)pks.mpg.de, https://www.pks.mpg.de/nqd

article: Friederike Metz and Marin Bukov, Self-correcting quantum many-body control using reinforcement learning with tensor networks, Nature Machine Intelligence 5, 780–791 (2023).

DOI: https://doi.org/10.1038/s42256-023-00687-5