An Overview of Multimodal Natural Language Processing Based on Artificial Intelligence: From Text Translation to Subject-Specific Analysis

In this review article, given that Multimodal Natural Language Processing (NLP) has made remarkable progress in the ability to convert multimedia inputs (text, image, audio) into each other, new architectures and solutions in the field of Multimodal NLP are examined; which include such things as translating text, audio, and image into each other, recognizing and generating image captions, and analyzing surrounding data. First, the architectures of convolutional neural networks, transformers, and various multimodal coding models are analyzed; then the advantages, challenges, and future research efforts are stated.

Keywords:

Multimodal NLP , Image Captioning , Artificial Intelligence , Machine Translation , Transformer