Reza Nouri - Researcher, Institute and Department of Artificial Intelligence and Cognitive Sciences ,IHU
Mohammadali Javadzadeh - Assistant Professor, Institute and Department of Artificial Intelligence and Cognitive Sciences, IHU

Artificial intelligence is advancing rapidly, powered by deep neural networks that enable capabilities like sentiment analysis. Sentiment analysis, a key natural language processing task, involves classifying the emotional valence within text. In this research, we utilize BERT (Bidirectional Encoder Representations from Transformers), one of the most powerful natural language processing algorithms, for sentiment analysis. Using the IMDB movie review dataset, we train a BERT model to classify movie reviews as expressing positive or negative sentiment. Our trained BERT model achieves ۸۹% accuracy on this binary sentiment classification task, demonstrating the impressive capabilities of BERT representations for understanding emotion in text. However, as a highly parameterized "black box" model, it remains unclear exactly how BERT arrives at these predictions. To peek inside the black box, we employ complex gradients attribution methods that assign importance scores to input words based on their influence on the model's output. This technique reveals which words BERT relies on for determining sentiment. We validate these importance scores using case studies on real IMDB reviews, confirming that BERT bases its predictions on relevant words that carry emotional valence. While BERT is highly performant, our analysis suggests potential areas for improvement. The word importance scores indicate that BERT sometimes neglects highly emotive words, while overvaluing seemingly inconsequential ones. future work could incorporate this insight into novel BERT-based architectures or training techniques. Overall, this research demonstrates BERT's utility for sentiment analysis, while also further demystifying BERT via attribution methods and case studies. Our approach illustrates how to not only benchmark neural network models, but also interpret their inner workings to guide progress.

Natural language processing, BERT, Explainability, Deep learning, Integrated gradiant, Sentiment analysis, Word importance