AI system learns to keep shop robot traffic efficient | MIT News

Inside a large private warehouse, hundreds of robots race through the aisles as they collect and distribute items to fill a continuous stream of customer orders. In this busy environment, even a traffic jam or a small collision can become a big snowball.

To avoid such inefficiencies, researchers from MIT and technology company Symbotic have developed a new method that automatically keeps a network of robots running smoothly. Their method learns which robots should go first each time, based on how congestion occurs, and adapts to prioritize robots that are about to get stuck. In this way, the system can reroute the robots in advance to avoid problems.

The hybrid system uses deep reinforcement learning, a powerful artificial intelligence method to solve complex problems, to determine which robots should be prioritized. Then, a fast and reliable programming algorithm feeds the robots commands, allowing them to respond quickly to constantly changing situations.

In measurements inspired by the real e-commerce design of the warehouse, this new method achieved about a 25 percent profit in the performance of other methods. Importantly, the system can quickly adapt to new situations with different numbers of robots or warehouse structures.

“There are many decision-making problems in production and management where companies rely on algorithms designed by human experts. But we have shown that, with the power of deep learning, we can achieve superhuman performance. This is a very promising method, because in these large warehouses even a 2 or 3 percent increase in throughput can have a significant impact on the System Laboratory of System Zheng,” said System Law (LIDS) at MIT and the lead author of the paper with this new method.

Zheng is joined on the paper by Yining Ma, LIDS postdoc; Brandon Araki and Jingkai Chen of Symbotic; and senior author Cathy Wu, Class of 1954 Associate Professor of Career Development in Civil and Environmental Engineering (CEE) and the Institute for Data, Systems, and Society (IDSS) at MIT, and a member of LIDS. The study appears today in the Journal of Artificial Intelligence Research.

Redirecting robots

Connecting hundreds of robots in an e-commerce warehouse simultaneously is not an easy task.

The problem is particularly complicated because the warehouse is a dynamic environment, and robots often find new jobs after achieving their goals. They need to be redirected quickly as they exit and enter the warehouse.

Companies often use algorithms written by human experts to decide where and when robots should move to maximize the number of packages they can handle.

But if there is congestion or conflict, the firm may have no choice but to close the entire warehouse for hours to resolve the issue manually.

“In this setting, we do not have an exact forecast of the future. We only know what the future may hold, in terms of incoming packages or the distribution of future orders. The planning system needs to keep up with these changes as warehousing operations continue,” Zheng said.

MIT researchers discovered this flexibility using machine learning. They started by designing a neural network model to look at the warehouse and decide how to prioritize the robots. They trained the model using deep reinforcement learning, a trial-and-error approach where the model learns to control robots in simulations that mimic real warehouses. The model is rewarded for making decisions that maximize overall performance while avoiding conflict.

Over time, the neural network learns to coordinate multiple robots correctly.

“By interacting with simulations inspired by real warehouse structures, our system receives feedback that we use to make its decisions smarter. A trained neural network can adapt to warehouses with different structures,” Zheng explained.

It is designed to capture long-term obstacles and obstacles in the path of each robot, while considering dynamic interactions between robots as they move through the warehouse.

By predicting current and future robot interactions, the model plans to avoid congestion before it happens.

After the neural network decides which robots should be prioritized, the system uses a tried-and-true scheduling algorithm to tell each robot how to get from one point to another. This efficient algorithm helps robots to react quickly to a changing material environment.

This combination of methods is important.

“This hybrid approach builds on my group’s work on how to achieve the best of both worlds between machine learning and classical development methods. Pure machine learning methods are still struggling to solve complex problems, yet it is very time-consuming and requires human experts to design efficient methods. But together, using expert-designed methods the right way can make the machine learning task much easier,” said the machine learning work.

Overcoming complexity

Once the researchers trained the neural network, they tested the system in simulated warehouses that were different from the ones they had seen during training. Since industrial simulations did not work well for this complex problem, researchers designed their own environments to simulate what happens in warehouses.

On average, their hybrid learning method achieved a 25 percent higher result than traditional algorithms and a random search method, in terms of the number of packages delivered by each robot. Their method may also generate possible robotic routing strategies that overcome the congestion caused by traditional methods.

“Especially when the density of robots in the warehouse increases, the difficulty increases significantly, and these traditional methods begin to break down quickly. In these areas, our method is very effective,” Zheng said.

Although their system is still far from real-world implementation, these demonstrations highlight the feasibility and benefits of using a machine learning-guided approach to warehouse automation.

In the future, researchers want to include task assignments in the formulation of the problem, as deciding which robot will complete each task affects congestion. They also plan to scale up their system to large warehouses with thousands of robots.

This study was sponsored by Symbotic.