Advancing Over-the-Air Federated Learning through Deep Reinforcement Learning in UAV-Assisted Networks with Movable Antennas

This paper investigates the deployment of over-the-air federated learning (OTA-FL), leveraging the dynamic repositioning and line-of-sight communication capabilities of unmanned aerial vehicles (UAVs) and movable antennas to enhance network efficiency. A closed-form expression is derived to quantify the optimality gap between the actual federated learning (FL) model and its theoretical ideal, accounting for the capabilities of movable antennas to show the diverse relationship between Mean Square Error (MSE) and the optimality gap. Then An MSE minimization problem is then formulated, involving the joint optimization of moveable antenna position vectors, and the beamforming vector at the UAV. This complex non-convex problem is reformulated as a Markov Decision Process (MDP) and solved using the Twin Delayed Deep Deterministic Policy Gradient (TD۳) algorithm within the deep reinforcement learning (DRL) framework. Numerical results demonstrate that the proposed algorithm outperforms benchmarks such as Advantage Actor-Critic(A۲C) and Soft Actor-Critic (SAC).

Keywords:

Over-the-air federated learning , Deep reinforcement learning , Unmanned aerial vehicles , Movable Antenna