Comparison of Q-learning and Sarsa Algorithm for Automated Guided Vehicle Path Planning

Abstract

An Automated Guided Vehicle (AGV) system is a type of material handling equipment that navigates through a facility using a combination of sensors and computer control. However, traditional path planning methods for AGVs often face challenges in determining efficient routes while ensuring obstacle avoidance and minimizing computational overhead. These limitations hinder the continuity and stabilization of production processes, particularly in complex and dynamic environment. This work explores path planning for AGVs based on reinforcement learning, specifically the Sarsa algorithm, where the AGV functions as an agent, influencing the continuity and stabilization of the production process. The problem is framed as a Markov Decision Process (MDP), allowing the AGV to model its environment and make sequential decisions to optimize its path. As the agent undergoes training, the emphasis gradually shifts towards exploitation rather than exploration. Problems involving obstacle avoidance strategies for static environments are also addressed, considering various learning rates, discount factors, and steps. Simulation results demonstrate that the AGV can avoid obstacles in a grid-mapped environment and reach its destination. Therefore, the Sarsa algorithm converges faster and requires fewer steps compared to Q-learning implementation.