Deep Reinforcement Learning for UAV Routing in the Presence of Multiple Charging Stations

Deploying Unmanned Aerial Vehicles (UAVs) for traffic monitoring has been a hotspot given their flexibility and broader view. However, a UAV is usually constrained by battery capacity due to limited payload. On the other hand, the development of wireless charging technology has allowed UAVs to replenish energy from charging stations. In this paper, we study a UAV routing problem in the presence of multiple charging stations (URPMCS) with the objective of minimizing the total distance traveled by the UAV during traffic monitoring. We present a deep reinforcement learning based method, where a multi-head heterogeneous attention mechanism is designed to facilitate learning a policy that automatically and sequentially constructs the route, while taking the energy consumption into account. In our method, two types of attentions are leveraged to learn the relations between monitoring targets and charging station nodes, adopting an encoder-decoder-like policy network. Moreover, we also employ a curriculum learning strategy to enhance generalization to different numbers of charging stations. Computational results show that our method outperforms conventional algorithms with higher solution quality (except for exact methods such as Gurobi) and shorter runtime in general, and also exhibits strong generalized performance on problem instances with different distributions and sizes.