Improving Knowledge Distillation Transfer Learning to Increase the Accuracy of the Lightweight Multi-Task Learning Classifier in Cognitive Software-Defined Radio

In the knowledge distillation transfer learning, due to the low capacity of the student model to capture transferred information gains from the teacher model, it is a challenge to increase the temperature coefficient (T) to achieve softer targets and more information gains. Also, inherent uncertainties in wireless communication data decrease the quality of the transmitted information. This paper proposes a solution to this challenge by applying active learning with fuzzy clustering sampling and fuzzy c-mean clustering on soft targets obtained from the temperature softmax activation function in an optimal temperature coefficient. This approach improves the quality of information transferred to the student model and, as a result, increases the accuracy of its performance. We implement the proposed learning approach on a lightweight classifier called the MobileNet V۳ MTL for simultaneous modulation and signal classification in cognitive software-defined radio (CSDR). The Fuzzy Active Knowledge Distillation (FAKD) learning method increases the model's generalizability. Then we compared the performance of the four extreme-edge devices, including SiFive FE ۳۱۰ (RV۳۲IMAC) and GD ۳۲VF۱۰۳ (RV۳۲IMAC) microcontrollers with RISC-V core architecture, and STM ۳۲L۴R۵ (ARM M) microcontroller and Raspberry Pi ۴B (ARM A) for RISC core architecture in the running model. Performance results from MobileNet V۳ MTL with the proposed learning approach on the RadComDynamic dataset show an improvement of ۶.۵۲% on the average classification accuracy of both modulation and signal in low SNR (-۴ dB), ۳.۵۵% in medium SNR (۰ dB), and ۱.۵۶% at high SNR (۱۰ dB) compared to training the model with vanilla knowledge distillation. Moreover, The evaluation results of running MobileNet V۳ MTL on MCUs show the superiority of RISC-V core architecture in the trade-off between real-time performance and ultra-low power consumption.

Keywords:

Cognitive Software-Defined Radio , Lightweight neural networks , Transfer learning , RISC-V Microcontrollers