EPFL and DeepMind use AI to control plasmas for nuclear fusion

EPFL’s Swiss Plasma Center (SPC) has decades of experience in plasma physics and plasma control methods. DeepMind is a scientific discovery company acquired by Google in 2014 that's committed to ‘solving intelligence to advance science and humanity. Together, they have developed a new magnetic control method for plasmas based on deep reinforcement learning, and applied it to a real-world plasma for the first time in the SPC’s tokamak research facility, TCV. Their study has just been published in Nature.

Tokamaks are donut-shaped devices for conducting research on nuclear fusion, and the SPC is one of the few research centers in the world that has one in operation. These devices use a powerful magnetic field to confine plasma at extremely high temperatures – hundreds of millions of degrees Celsius, even hotter than the sun’s core – so that nuclear fusion can occur between hydrogen atoms. The energy released from fusion is being studied for use in generating electricity. What makes the SPC’s tokamak unique is that it allows for a variety of plasma configurations, hence its name: variable-configuration tokamak (TCV). That means scientists can use it to investigate new approaches for confining and controlling plasmas. A plasma’s configuration relates to its shape and position in the device.

Controlling a substance as hot as the Sun

Tokamaks form and maintain plasmas through a series of magnetic coils whose settings, especially voltage, must be controlled carefully. Otherwise, the plasma could collide with the vessel walls and deteriorate. To prevent this from happening, researchers at the SPC first test their control systems configurations on a simulator before using them in the TCV tokamak. “Our simulator is based on more than 20 years of research and is updated continuously,” says Federico Felici, an SPC scientist and co-author of the study. “But even so, lengthy calculations are still needed to determine the right value for each variable in the control system. That’s where our joint research project with DeepMind comes in.”

3D model of the TCV vacuum vessel containing the plasma, surrounded by various magnetic coils to keep the plasma in place and to affect its shape. Credit: DeepMind & SPC/EPFL

DeepMind’s experts developed an AI algorithm that can create and maintain specific plasma configurations and trained it on the SPC’s simulator. This involved first having the algorithm try many different control strategies in simulation and gathering experience. Based on the collected experience, the algorithm generated a control strategy to produce the requested plasma configuration. This involved first having the algorithm run through a number of different settings and analyze the plasma configurations that resulted from each one. Then the algorithm was called on to work the other way – to produce a specific plasma configuration by identifying the right settings. After being trained, the AI-based system was able to create and maintain a wide range of plasma shapes and advanced configurations, including one where two separate plasmas are maintained simultaneously in the vessel. Finally, the research team tested their new system directly on the tokamak to see how it would perform under real-world conditions.

Range of different plasma shapes generated with the reinforcement learning controller
Credit: DeepMind & SPC/EPFL

The SPC’s collaboration with DeepMind dates back to 2018 when Felici first met DeepMind scientists at a hackathon at the company’s London headquarters. There he explained his research group’s tokamak magnetic-control problem. “DeepMind was immediately interested in the prospect of testing their AI technology in a field such as nuclear fusion, and especially on a real-world system like a tokamak,” says Felici. Martin Riedmiller, control team lead at DeepMind and co-author of the study, adds that “our team’s mission is to research a new generation of AI systems – closed-loop controllers – that can learn in complex dynamic environments completely from scratch. Controlling a fusion plasma in the real world offers fantastic, albeit extremely challenging and complex, opportunities.”

A win-win collaboration

After speaking with Felici, DeepMind offered to work with the SPC to develop an AI-based control system for its tokamak. “We agreed to the idea right away, because we saw the huge potential for innovation,” says Ambrogio Fasoli, the director of the SPC and a co-author of the study. “All the DeepMind scientists we worked with were highly enthusiastic and knew a lot about implementing AI in control systems.” For his part, Felici was impressed with the amazing things DeepMind can do in a short time when it focuses its efforts on a given project.

DeepMind also got a lot out of the joint research project, illustrating the benefits to both parties of taking a multidisciplinary approach. Brendan Tracey, a senior research engineer at DeepMind and co-author of the study, says: “The collaboration with the SPC pushes us to improve our reinforcement learning algorithms, and as a result can accelerate research on fusing plasmas.”

This project should pave the way for EPFL to seek out other joint R&D opportunities with outside organizations. “We’re always open to innovative win-win collaborations where we can share ideas and explore new perspectives, thereby speeding the pace of technological development,” says Fasoli.