Optimal Naval Movement Simulation with Reinforcement Learning AI Agents

As the U.S. Navy and its allies strive to ensure freedom of the seas, the need for effective strategies in naval engagements is paramount. Despite expectations, instances such as the USS John McCain and USS Fitzgerald have shown that identifying advantageous moves in every interaction can be challenging. Leveraging advancements in machine learning (ML) and artificial intelligence (AI), this study developed a simulation-based program that applies reinforcement learning (RL) to naval scenarios. The program, an adaptation of an existing land-based wargaming simulation, Atlatl, was designed to identify efficient movements for own forces in six scenarios. Evaluations of Deep Q-Network (DQN), Monte Carlo tree search (MCTS), and AlphaStar AI agents across various scenarios revealed that DQN and MCTS agents were able to identify superior strategies, with DQN demonstrating consistently high scores and outperforming human players in some scenarios. AlphaStar showed fewer promising results but provided insight into how it can be altered for better results in the future. These findings underscore the potential of AI as a decision aid in naval operations, contributing to enhanced decision-making in the U.S. Navy. Future research is recommended to further explore this potential.