Applying Machine Learning to Twitter and News Data to Predict Topics that Politicians will Tweet about on a Daily Basis
Abstract:
In recent years, social media has become an integral part of political discussions. President Trump has been a major user of Twitter, and concerns have been raised about how unpredictable he can seem on the platform. In response to politicians' increased use of social media, this research attempts to use machine learning to effectively predict what topics 20 diverse politicians on Twitter will tweet about each day. The data set used was collected from Twitter and RSS news headlines, and was run through Amazon Web Services Comprehend to generate 20 topics for each of 82 days examined. Then, a 2 layer neural network was implemented to predict whether the politicians would tweet about each topic. Contrary to expectations, predictions for President Trump were second best, after Trump Jr., with an accuracy of 66.77%, precision of 67.84%, recall of 72.05%, and F1 score of 69.88%, notably outperforming predictions for President-elect Biden. Predictions were ineffective on some politicians such as President Obama and Senator Romney, likely due to their lower usage of Twitter. Additionally, tests to see if one political party was more influential in predicting Trump’s and Biden’s tweets had insignificant results. These predictions of future tweets’ topics can be used to create forecasts of politicians actions and avoid being caught unawares by what they choose to tweet about. In the future, models similar to that of this project could be used for comprehensive predictions of all the major aspects of tweets, such as topic and sentiment.
If your interested you can read the full paper here: bit.ly/JMPresearch
Flow chart of research process
Table of results