Applying Domain Layer Normalization on Long-Short Term Memory network for adaptive Classification

Recurrent based classification models often face challenges in achieving high performance when provided with a limited amount of data. In such scenarios, if we have data for different related tasks, multi-task learning could be a useful approach for leveraging their commonalities and differences to enhance the overall performance. Many existing works in multi-task classification apply complex architecture for sharing related information and handling differences. In this study, we introduce an innovative approach to address this issue by incorporating layer normalization into the model’s design, enabling effective control of information flow. We refer to our model as Domain Layer Norm Classifier (DLNC). Additionally, we define task-specific embeddings to further characterize each task and enhance performance. Despite its simplicity and lightweight design, our model demonstrates robust generalization capabilities, even when trained on limited data. To validate our approach, we conduct fine-tuning experiment on a new dataset, highlighting the adaptability and effectiveness of our model.

Keywords:

Classification , layer normalization , multi-task learning , Recurrent neural networks , task embedding , fine- tuning