An Efficient Autonomous Exploration for Unknown Environment Mapping via Deep Reinforcement Learning

This research explores the integration of deep reinforcement learning (DRL) and attention mechanisms in exploration planning, aiming to enhance autonomous exploration and mapping in unknown environments. Traditional exploration strategies often struggle with inefficiencies, such as redundant revisits and suboptimal global paths. To overcome these challenges, a novel exploration sequence planner is proposed, which replaces the heuristic asymmetric traveling salesman problem (ATSP) solver in FAEP with a deep deterministic policy gradient (DDPG) method augmented with an attention mechanism. This approach enables the agent to dynamically learn efficient exploration sequences by considering spatial features of frontiers and its proximity to unexplored areas. Through simulation experiments, the proposed method demonstrates improvements in exploration efficiency, reducing exploration time and flight distance compared to state-of-the-art benchmarks, including FAEP and FUEL. The results underscore the potential of DRL and attention mechanisms in advancing autonomous robotic exploration, paving the way for the development of more intelligent and adaptive systems in various applications, such as surveying, rescue operations, and ۳D reconstruction.

Keywords:

Deep Reinforcement Learning , Attention Mechanism , Autonomous Exploration , Deterministic Policy Gradient