Deep Reinforcement Learning Improves Automatic Routing Performance in OTNs

Researchers from Universitat Politècnica de Catalunya and Huawei have retooled an AI technique to make optical transport networks (OTNs) run more efficiently. OTNs could be used to package data in the fiber optic cables used for transmitting data over long distances, and improving OTN performance would make this transmission more efficient.

OTNs require rules to make the split-second decisions required to manage large amounts of traffic. In a new approach to writing these rules, the researchers combined two machine learning techniques: reinforcement learning and deep learning. In reinforcement learning, a virtual “agent” learns through trial and error to optimize how resources are managed. Deep learning supports the reinforcement-based approach by using neural networks to draw more abstract conclusions from each round of trial and error.

So far, the most advanced deep reinforcement learning algorithms have been able to optimize some resource allocation in OTNs, but become stuck when they run into novel scenarios. The researchers worked to overcome this by varying the manner in which data are presented to the agent. After learning the OTNs through 5000 rounds of simulations, the researchers found that the deep reinforcement learning agent directed traffic with 30 percent greater efficiency than the current state-of-the-art algorithm.

One thing that surprised the team was how easily the new approach was able to learn about the networks after starting out with a blank slate. “This means that without prior knowledge, a deep reinforcement learning agent can learn how to optimize a network autonomously,” professor Albert Cabellos-Aparicio said. “This results in optimization strategies that outperform expert algorithms.” With the enormous scale some optical transport networks already have, Cabellos-Aparicio said, even small advances in efficiency could reap large returns in reduced latency and operational costs.

Next, the team plans to apply its deep reinforcement learning strategies in combination with graph networks, an emerging field within artificial intelligence with the potential to transform the computer networks, chemistry, and logistics used in industry and science.

The team’s technical results are outlined in a paper presented at OFC: The Optical Fiber Communications Conference and Exhibition, March 3-7, in San Diego (https://doi.org/10.1364/OFC.2019.M2A.6).