With the popularity of social networks and e-commerce sites, users have switched from passively receiving messages to actively disseminating messages. The value of comments and online messages is also becoming more and more important. Analysis and research over the past few years. Trying to analyze trends about specific product products, topics, reviews, and tweets. Play an important role in all aspects. This study uses different vectorization processes to verify the multimodal analysis model and confirm that the model can effectively improve the accuracy. This study proposes a combination of two models. This feature is combined with deep learning neural network construction to build a multimodal analysis model. Model 1 is a deep learning model based on Glove vector, attention mechanism and GRU neural network architecture. Model 2 is a deep learning model based on Word2Vec vector, attention mechanism and CNN neural network architecture. Multimodal analysis model is validated by K-fold cross validation and F1 measurement method. The experimental results prove that the multimodal analysis model proposed in this study has higher accuracy than related research. Using the high-level multi-modal combination method, the features of multiple models are extracted and combined to form a combined feature, and this feature is trained in neural network. The feature set can be mutually assisted, and the accuracy can be 91.56% through the two vectors and the optimal neural network architecture combined with the multi-modal method. And the model verification shows 93% verification value, which proves that the multimodal analysis model proposed in this study is used in the field of comment texts, which can effectively improve the accuracy of model prediction and improve its accuracy.
摘要 i
Abstract ii
誌謝 iii
目錄 iv
圖目錄 vii
表目錄 ix
第一章、緒論 1
1-1 研究背景與動機 1
1-2 研究目的 2
1-3 論文架構 3
第二章、文獻探討 4
2-1 詞向量 4
2-1-1 Word2Vec 4
2-1-2 全局向量 6
2-2 類神經網路 6
2-2-1 卷積神經網路 7
2-2-1-1 卷積層 8
2-2-1-2 池化層 8
2-2-1-3 全連接層 9
2-2-2 長短期記憶網路 10
2-2-3 GRU 12
2-3 激活函數 13
2-3-1 Sigmoid 13
2-3-2 ReLU 14
2-4 注意力機制 16
2-5 多模態深度學習 17
2-6 K折交叉驗證 20
2-7 F1測量驗證 21
2-8 準確度驗證 22
第三章、研究方法與架構 23
3-1 實驗架構 23
3-2 實驗準備 25
3-3 實驗比較對象 26
3-3-1 實驗比較對象一 26
3-3-2 實驗比較對象二 28
3-4 實驗流程 30
3-4-1 前置實驗 30
3-4-1-1 詞向量訓練 31
3.4.1.1.1. Word2Vec字詞模型訓練 31
3.4.1.1.2. Glove字詞模型訓練 31
3-4-2 實驗一 32
3-4-2-1 GRU與長短期記憶網路模型建構 32
3-4-2-2 CNN模型建構 33
3-4-2-3 注意力機制 34
3-4-3 實驗二 35
3-4-3-1 多模態特徵結合與神經網路建構 35
第四章、實驗結果 37
4-1 實驗一結果 37
4-2 實驗二結果 42
4-3 實驗總結 45
第五章、研究結論 46
5-1 結論 46
5-2 研究貢獻 46
5-3 研究限制 47
5-4 未來研究方向 47
參考文獻 48
附錄一: 模型一程式碼 51
附錄二: 模型二程式碼 59
附錄三: 多模態分析模型程式碼 65
Andrew. (2011). Learning word vectors for sentiment analysis.
Arras & Montavon. (2016). Explaining predictions of non-linear classifiers in NLP.
Azimi & Abdolrashidi. (2019). Deep-Sentiment: Sentiment Analysis Using Ensemble of CNN and Bi-LSTM Models.
Bagnall, A., Lines, J., Hills, J., & Bostrom, A. (2015). Time-series classification with COTE: the collective of transformation-based ensembles.
Bargal&Sclaroff. (2018). Top-down neural attention by excitation backprop.
Bengio & Grandvalet. (2014). No unbiased estimator of the variance of k-fold cross-validation.
Cerisara & Lenc. (2018). On the effects of using word2vec representations in neural networks for dialogue act recognition.
Chua&Sun. (2015). Topical word embeddings.
Dayan & Abbott. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems.
Dhariyal & Ravi. (2018). Sentiment analysis via Doc2Vec and Convolutional Neural Network hybrids.
Hansen & Simonsen. (2019). Neural Speed Reading with Structural-Jump-LSTM.
Hinton&Salakhutdinov. (2006). Reducing the dimensionality of data with neural networks.
Hochreiter & Schmidhuber. (1997). Long short-term memory. Neural computation.
Ji, L., Gong, P., & Yao, Z. (2019). A text sentiment analysis model based on self-attention mechanism.
Kai Sheng Tai & Richard Socher. (2015). Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks.
Khosla & Ng. (2011). Multimodal deep learning.
KimY. (2014). Convolutional neural networks for sentence classification.
Kingma & Ba. (2014). Adam: A method for stochastic optimization.
Koley & Dey. (2012). An ensemble system for automatic sleep stage classification using single channel EEG signal.
Lines & Bostrom. (2017). Time series classification from scratch with deep neural networks: A strong baseline.
Liu & Xiong. (2018). Attention Aware Bidirectional Gated Recurrent Unit Based Framework for Sentiment Analysis.
Mikolov & Dean. (2013). Efficient estimation of word representations in vector space.
Nanopoulos & Manolopoulos. (2001). Feature-based classification of time-series data.
Peng & Zhao. (2017). Object-part attention model for fine-grained image classification.
Pennington & Manning. (2014). Glove: Global vectors for word representation.
Ravi&Dhariyal. (2018). Sentiment analysis via Doc2Vec and Convolutional Neural Network hybrids.
Ren&Bao. (2018). Investigating Lstm with k-Max Pooling for Text Classification.
RosenblattF. (1958). The perceptron: a probabilistic model for information storage and organization in the brain.
Shazeer & Jones. (2017). Attention is all you need.
Simard & Frasconi. (1994). Learning long-term dependencies with gradient descent is difficult.
Subarno & Ghosh. (2018). Sentiment Analysis in the Light of LSTM Recurrent Neural Networks.
Sutskever & Hinton. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems.
Torralba&Fidler. (2015). Skip-Thought Vectors.
Tsuruoka. (2016). A joint many-task model: Growing a neural network for multiple nlp tasks.
Wei & Keogh. (2006). Semi-supervised time series classification.
Williams & Zipser. (1989). A learning algorithm for continually running fully recurrent neural networks.
Xianghua. (2018). Lexicon-enhanced LSTM with attention for general sentiment analysis.
Xiao & Zhao. (2018). A deep learning-based multi-model ensemble method for cancer prediction.
Xinpeng & Jingyuan. (2018). Fine-grained Video Attractiveness Prediction Using Multimodal.
Zhicheng Cui. (2016). Multi-scale convolutional neural networks for time series classification.