S&P۵۰۰ Index Direction Prediction Using Textual Tweets and Their Corresponding Sentiment

In this paper, a novel method is proposed to predict the direction of Standard & Poor ۵۰۰ (S&P۵۰۰) index using the tweets in this regard as well as the index amount from the day before. At the beginning, using a dataset of all tweets and their corresponding posting times about S&P۵۰۰ index, companies and securities are considered as features of the study. Next, these feature vectors are assigned three different labels based on the direction of the index change from the day before and whether the change is significant enough, creating a classification problem. Building a sentiment analysis tool based on T۵ transformer which attempts to combine all the downstream tasks into a text-to-text format, sentiment feature is added to each tweet in the dataset. Lastly, after balancing the data and preprocessing the textual information through an NLP pipeline, a deep neural network is proposed to classify the processed data. It is shown that using the tweets and their corresponding sentiments, the proposed method for movement direction prediction of the S&P۵۰۰ index outperformed other existing models.

Keywords:

Natural Language Processing , Predictive Modeling , Sentiment analysis , deep neural networks , Transformer Learning