A Comparative Review of Multilingual Language Models: Evaluation of mBERT and XLM-R on Cross-Lingual Tasks

Multilingual pre-trained models have revolutionized natural language processing (NLP) by enabling cross-language transfer and enhancing performance in low-resource languages. This review article examines three fundamental studies on multilingual language models: the XTREME benchmark for evaluating cross-language generalization, the XLM-R architecture with its large-scale training, and a comparative evaluation of multilingual advertisement recognition using mBERT, XLM-R, and mT۵. We summarize the methodologies, datasets, evaluation tasks, and results from these studies, and provide a hybrid comparison that highlights the strengths and limitations of mBERT and XLM-R across various NLP tasks.

Keywords:

Natural Language Processing , Multilingual advertisement recognition , XTREME benchmark , mBERT