Integration of Q-learning and Sarsa Algorithm for Automated Guided Vehicle Path Planning

Abstract

An Automated Guided Vehicle (AGV) system is a type of material handling equipment that navigates through a facility using a combination of sensors and computer control. Path planning for AGV systems involves determining the most efficient route for the AGV to travel. This work explores path planning for AGVs based on reinforcement learning, specifically the Sarsa algorithm, where the AGV functions as an agent, influencing the continuity and stabilization of the production process. The problem is framed as a Markov Decision Process (MDP), allowing the AGV to model its environment and make sequential decisions to optimize its path. As the agent undergoes training, the emphasis gradually shifts towards exploitation rather than exploration. Problems involving obstacle avoidance strategies for static environments are also addressed, considering various learning rates, discount factors, and steps. Simulation results demonstrate that the AGV can successfully avoid obstacles in a grid-mapped environment and reach its destination. Therefore, the Sarsa algorithm converges faster and requires fewer steps compared to Q-learning implementation.