LSTM-based sentiment analysis for stock price forecast.
Ontology highlight
ABSTRACT:
SUBMITTER: Ko CR
PROVIDER: S-EPMC7959635 | biostudies-literature | 2021
REPOSITORIES: biostudies-literature
Publications
LSTM-based sentiment analysis for stock price forecast.
Ko Ching-Ru CR Chang Hsien-Tsung HT
PeerJ. Computer science 20210311
Investing in stocks is an important tool for modern people's financial management, and how to forecast stock prices has become an important issue. In recent years, deep learning methods have successfully solved many forecast problems. In this paper, we utilized multiple factors for the stock price forecast. The news articles and PTT forum discussions are taken as the fundamental analysis, and the stock historical transaction information is treated as technical analysis. The state-of-the-art natu ...[more]
Similar Datasets
Project description:Correctly predicting the stock price movement direction is of immense importance in the financial market. In recent years, with the expansion of dimension and volume in data, the nonstationary and nonlinear characters in finance data make it difficult to predict stock movement accurately. In this article, we propose a methodology that combines technical analysis and sentiment analysis to construct predictor variables and then apply the improved LASSO-LASSO to forecast stock direction. First, the financial textual content and stock historical transaction data are crawled from websites. Then transfer learning Finbert is used to recognize the emotion of textual data and the TTR package is taken to calculate the technical indicators based on historical price data. To eliminate the multi-collinearity of predictor variables after combination, we improve the long short-term memory neural network (LSTM) model with the Absolute Shrinkage and Selection Operator (LASSO). In predict phase, we apply the variables screened as the input vector to train the LASSO-LSTM model. To evaluate the model performance, we compare the LASSO-LSTM and baseline models on accuracy and robustness metrics. In addition, we introduce the Wilcoxon signed rank test to evaluate the difference in results. The experiment result proves that the LASSO-LSTM with technical and sentiment indicators has an average 8.53% accuracy improvement than standard LSTM. Consequently, this study proves that utilizing historical transactions and financial sentiment data can capture critical information affecting stock movement. Also, effective variable selection can retain the key variables and improve the model prediction performance.
| S-EPMC9680880 | biostudies-literature
Project description:Social media are increasingly reflecting and influencing behavior of other complex systems. In this paper we investigate the relations between a well-known micro-blogging platform Twitter and financial markets. In particular, we consider, in a period of 15 months, the Twitter volume and sentiment about the 30 stock companies that form the Dow Jones Industrial Average (DJIA) index. We find a relatively low Pearson correlation and Granger causality between the corresponding time series over the entire time period. However, we find a significant dependence between the Twitter sentiment and abnormal returns during the peaks of Twitter volume. This is valid not only for the expected Twitter volume peaks (e.g., quarterly announcements), but also for peaks corresponding to less obvious events. We formalize the procedure by adapting the well-known "event study" from economics and finance to the analysis of Twitter data. The procedure allows to automatically identify events as Twitter volume peaks, to compute the prevailing sentiment (positive or negative) expressed in tweets at these peaks, and finally to apply the "event study" methodology to relate them to stock returns. We show that sentiment polarity of Twitter peaks implies the direction of cumulative abnormal returns. The amount of cumulative abnormal returns is relatively low (about 1-2%), but the dependence is statistically significant for several days after the events.
| S-EPMC4577113 | biostudies-literature
Project description:Investor sentiment plays a crucial role in the stock market, and in recent years, numerous studies have aimed to predict future stock prices by analyzing market sentiment obtained from social media or news. This study investigates the use of investor sentiment from social media, with a focus on Stocktwits, a social media platform for investors. However, using investor sentiment on Stocktwits to predict stock price movements may be challenging due to a lack of user-initiated sentiment data and the limitations of existing sentiment analyzers, which may inaccurately classify neutral comments. To overcome these challenges, this study proposes an alternative approach using FinBERT, a pre-trained language model specifically designed to analyze the sentiment of financial text. This study proposes an ensemble support vector machine for improving the accuracy of stock price movement predictions. Then, it predicts the future movement of SPDR S&P 500 Index Exchange Traded Funds using the rolling window approach to prevent look-ahead bias. Through comparing various techniques for generating sentiment, our results show that using the FinBERT model for sentiment analysis yields the best results, with an F1-score that is 4-5% higher than other techniques. Additionally, the proposed ensemble support vector machine improves the accuracy of stock price movement predictions when compared to the original support vector machine in a series of experiments.
| S-EPMC10280432 | biostudies-literature
Project description:Crowd management refers to the management and control of masses at specific locations. A Hajj gathering is an example. Hajj is the biggest gathering of Muslims worldwide. Over two million Muslims from all over the globe come annually to Makkah, Saudi Arabia. Authorities of Saudi Arabia strive to provide comfortable comprehensive services to pilgrims using the latest modern technologies. Recent studies have focused on camera scenes and live streaming to assess the count and monitor the behavior of the crowd. However, the opinions of the pilgrims and their feelings about their experience of Hajj are not well known, and the data on social media (SM) is limited. This paper provides a combination of Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) algorithms for sentiment analysis of pilgrims using a novel and specialized dataset, namely Catering-Hajj. The model is based on four CNN layers for local feature extraction after the One-Hot Encoder, and one LSTM layer to maintain long-term dependencies. The generated feature maps are passed to the SoftMax layer to classify final outputs. The proposed model is applied to a real case study of issues related to pre-prepared food at Hajj 1442. Started with collecting the dataset, extracting target attitudes, annotating the data correctly, and analyzing the positive, negative, and neutral attitudes of the pilgrims to this event. Our model is compared with a set of Machine Learning (ML) models including Support Vector Machine (SVM), Logistic Regression (LR), Decision Tree (DT), and Random Forest (RF), as well as CNN and LSTM models. The experimental results show that SVM, RF, and LSTM achieve the same rate of roughly 81%. LR and CNN achieve 79%, and DT achieves 71%. The proposed model outperforms other classifiers on our dataset by 92%.
| S-EPMC11784790 | biostudies-literature
Project description:A significant correlation between financial news with stock market trends has been explored extensively. However, very little research has been conducted for stock prediction models that utilize news categories, weighted according to their relevance with the target stock. In this paper, we show that prediction accuracy can be enhanced by incorporating weighted news categories simultaneously into the prediction model. We suggest utilizing news categories associated with the structural hierarchy of the stock market: that is, news categories for the market, sector, and stock-related news. In this context, Long Short-Term Memory (LSTM) based Weighted and Categorized News Stock prediction model (WCN-LSTM) is proposed. The model incorporates news categories with their learned weights simultaneously. To enhance the effectiveness, sophisticated features are integrated into WCN-LSTM. These include, hybrid input, lexicon-based sentiment analysis, and deep learning to impose sequential learning. Experiments have been performed for the case of the Pakistan Stock Exchange (PSX) using different sentiment dictionaries and time steps. Accuracy and F1-score are used to evaluate the prediction model. We have analyzed the WCN-LSTM results thoroughly and identified that WCN-LSTM performs better than the baseline model. Moreover, the sentiment lexicon HIV4 along with time steps 3 and 7, optimized the prediction accuracy. We have conducted statistical analysis to quantitatively assess our findings. A qualitative comparison of WCN-LSTM with existing prediction models is also presented to highlight its superiority and novelty over its counterparts.
| S-EPMC9990937 | biostudies-literature
Project description:This paper constructs the measurement index of core competence by text analysis method and empirically tests the impact of core competence on stock price synchronicity. We find that the stronger the core competence, the lower the stock price synchronicity, and mechanism test shows that core competence reduces the stock price synchronicity by enhancing the transparency of corporate information, which is still valid under a series of robustness tests. Further research shows that:(1) when the corporate governance environment is poor (higher level of internal earnings management, lower quality of accounting information, greater separation of ownership and control, lower shareholding ratio of external institutional investors, weaker product market competition, less media attention), the core competence has a more significant effect on the decline of stock price synchronization; (2)vertically, the dynamic improvement of core competence in the time dimension can play a role in stabilizing stock price synchronization; (3)after distinguishing the types of core competence, we find that the core competence related to information disclosure is more helpful to reduce the stock price synchronization; (4)after the CSRC forces listed companies to disclose the core competence information in the annual report, the core competence plays a stronger role in reducing the stock price synchronization. This study reveals the important role of core competence in reducing stock price synchronization. It not only enriches the relevant literature of core competence and stock price synchronization, but also has important practical significance for the government and regulatory departments to improve the efficiency of capital market allocation.
| S-EPMC8577773 | biostudies-literature
Project description:The recent COVID-19 pandemic represents an unprecedented worldwide event to study the influence of related news on the financial markets, especially during the early stage of the pandemic when information on the new threat came rapidly and was complex for investors to process. In this paper, we investigate whether the flow of news on COVID-19 had an impact on forming market expectations. We analyze 203,886 online articles dealing with COVID-19 and published on three news platforms (MarketWatch.com, NYTimes.com, and Reuters.com) in the period from January to June 2020. Using machine learning techniques, we extract the news sentiment through a financial market-adapted BERT model that enables recognizing the context of each word in a given item. Our results show that there is a statistically significant and positive relationship between sentiment scores and S&P 500 market. Furthermore, we provide evidence that sentiment components and news categories on NYTimes.com were differently related to market returns.
| S-EPMC9842392 | biostudies-literature
Project description:The stock market is very complex and volatile. It is impacted by positive and negative sentiments which are based on media releases. The scope of the stock price analysis relies upon ability to recognise the stock movements. It is based on technical fundamentals and understanding the hidden trends which the market follows. Stock price prediction has consistently been an extremely dynamic field of exploration and research work. However, arriving at the ideal degree of precision is still an enticing challenge. In this paper, we are proposing a combined effort of using efficient machine learning techniques coupled with a deep learning technique-Long Short Term Memory (LSTM)-to use them to predict the stock prices with a high level of accuracy. Sentiments derived by users from news headlines have a tremendous effect on the buying and selling patterns of the traders as they easily get influenced by what they read. Hence, fusing one more dimension of sentiments along with technical analysis should improve the prediction accuracy. LSTM networks have proved to be a very useful tool to learn and predict temporal data having long term dependencies. In our work, the LSTM model uses historical stock data along with sentiments from news items to create a better predictive model.
| S-EPMC7924447 | biostudies-literature
Project description:Livestock product prices serve as a barometer and bellwether for the agricultural market. However, traditional point prediction techniques focus mainly on tracking or fitting, resulting in limited information and challenges in evaluating the uncertainty of future prices. A comprehensive livestock price prediction model with joint point and interval prediction capabilities is proposed, with fuzzy mathematics and long short-term memory. Three main steps are taken: (1) data composition and reconstruction, to extract a set of relatively stationary subsequence components by complementary ensemble empirical mode decomposition (CEEMD) from original signal, and divide these components into three groups according to fuzzy entropy (FE) value. (2) characteristics categorization, determining the lower bound, mean, and upper bound of the rebuilt data via fuzzy information granulation (FIG) to better characterize the price fluctuation range. (3) price prediction, including point and interval predictions with attention mechanism long short-term memory (AM-LSTM). An empirical study was conducted on the weekly price data of pork, beef, and mutton in China from 2009 to 2023, incorporating discussions on different embedding dimensions, prediction step, fuzzy granulation window sizes, decomposition techniques, and prediction algorithms. The results indicate that the proposed interval prediction model can not only achieve high accuracy in point prediction, but also better capture price change intervals.
| S-EPMC11828389 | biostudies-literature
Project description:The literature provides strong evidence that stock price values can be predicted from past price data. Principal component analysis (PCA) identifies a small number of principle components that explain most of the variation in a data set. This method is often used for dimensionality reduction and analysis of the data. In this paper, we develop a general method for stock price prediction using time-varying covariance information. To address the time-varying nature of financial time series, we assign exponential weights to the price data so that recent data points are weighted more heavily. Our proposed method involves a dimension-reduction operation constructed based on principle components. Projecting the noisy observation onto a principle subspace results in a well-conditioned problem. We illustrate our results based on historical daily price data for 150 companies from different market-capitalization categories. We compare the performance of our method to two other methods: Gauss-Bayes, which is numerically demanding, and moving average, a simple method often used by technical traders and researchers. We investigate the results based on mean squared error and directional change statistic of prediction, as measures of performance, and volatility of prediction as a measure of risk.
| S-EPMC7083277 | biostudies-literature