Optimizing Naval Movement Using Deep Reinforcement Learning

In a rapidly evolving maritime warfare landscape, the U.S. Navy and its allies require their crews to quickly identify optimal strategies for vessel engagements to ensure freedom of the seas. This necessity becomes more pronounced given the potential grave consequences of sub-optimal maneuvers, as illustrated by the cases of the USS John McCain and USS Fitzgerald. Recent advancements in Machine Learning (ML) and Artificial Intelligence (AI) offer a promising solution. There have been significant strides in implementing AI to outperform human experts in complex games such as Chess, Poker, and StarCraft, which now have the potential to also benefit real-time decision-making and wargaming in the naval domain. This study explores the potential for Reinforcement Learning (RL) techniques to be applied to naval contexts, which could provide valuable decision support tools to ship captains and their staff by suggesting optimal movement strategies in complex maritime scenarios. In this study, exemplar naval scenarios were designed and modeled within a combat simulation environment, AI agents (consisting of a mix of rule-based, method-based, and value-based approaches) were designed, and the performances of these agents were evaluated and compared. The aim was to assess the agents' ability to identify optimal movements against a rule-based adversary, while also comparing these performances against human-level play. The insights drawn from this study contribute to ongoing research aimed at developing effective decision aids for ship captains in real-world operations.