Stock Volume Prediction Based on Social Sentiments, News and Historical Data

Machine Learning and Natural Language Processing are branches of modern technology that are fast being used to solve myriads of problems that inflict us in our daily life. A particular problem that has the potential to benefit greatly from the capabilities of data mining and machine learning is the issue of unpredictability in the stock market environment. What makes this a difficult problem is that stock volume movements are influenced by a variety of factors some of which are inherently quantifiable while others such as trader sentiments are not. The system proposed in this paper combines Fundamental and Technical trading philosophies in predicting stock volume movements, during day trading, based on historical stock performance data, financial news, and trader sentiments. Financial news articles, for a stipulated time period, are collected and filtered based on the companies mentioned in the articles. For this paper, we have chosen to filter and retain articles about companies belonging to SENSEX 50. For gauging trader sentiments with respect to the news about a company or the company in general, Twitter tweets are considered as a data source. Sentiment analysis is performed for the news and cumulated tweets for a company separately to arrive at two polarity scores that indicate the sentiments carried. A regression model is developed that takes as input these polarity scores and OHLC (Open, High, Low, Close) to predict the stock volume movement.