A Comparative analysis of conventional PID, Q-Learning and DQN approaches; A flight control example

This research examines the comparative efficiency of proportional-integral-derivative (PID) controllers, adaptive PID controllers based on Q-Learning, and deep Q network (DQN) approaches for the regulation of the theta angle in passenger aircraft. The adaptive PID controller employs a Q-Learning simulation to dynamically adjust PID coefficients derived from Q-tables in response to the current system state. The objective of this theta angle stabilization technique is to facilitate a transition from a non-zero initial condition to an equilibrium state, demonstrating considerable superiority over classical controller strategies such as the PID controller. Despite its limitations in the face of uncertainties or the introduction of noise into the system, the application of artificial intelligence algorithms for parameter control is anticipated to yield enhanced performance, albeit at the cost of increased computational demands. Controllers based on artificial intelligence algorithms exhibit an exceptional capability to manage a wide range of initial conditions, a trait similarly exhibited by the DQN approach. In the DQN controller, the coefficients are processed through a neural network, with the output being the system error, which is subsequently evaluated and rewarded by the DQN algorithm. However, unlike direct control via DQN, the adaptive PID controller utilizes Q-tables for coefficient adjustment, thereby enhancing computational efficiency and expediting the learning process. This paper underscores the predominant advantages of the adaptive PID controller and rapid adaptation speed, positioning it as a superior option for the control of aircraft theta angles.