現今人工智慧已逐漸改變人類對電腦的使用習慣，透過生成式互動問答即可獲取資訊。此行為使用模式的轉變，源於大型語言模型具備處理和理解大量文本資料技術的快速發展，從中找出關鍵字、研究熱點和技術趨勢，為研究者提供發現學術動向的途徑。而主題模型能夠有效地將大量文本資料轉換為具體主題，通過分析文章的分佈或矩陣分解，了解文本內容分類與關鍵字主題，提升資料檢索效率，並動態追蹤研究領域發展與變化，探索研究方向和關鍵字熱度。隨著時間推移，主題模型能夠反映學術研究的最新進展與發展。然而，現實中資料或主題會隨時間推移出現、更新或消失，分析主題與時間的關聯性，即為本文探討的主題模型重要性。
本研究以大型語言模型中為主題透過深度學習計算頻率最高的前十大關鍵字，觀察分析主題熱度，說明主題的重要性。這些主題不僅代表技術發展的趨勢，並可說明研究重點的轉變。通過對主題關鍵字的深入分析，我們可以更全面地快速理解研究領域發展歷程及其重要主題分析中的影響變化。本研究系統分析arXiv資料庫2007年至2023年以LLM為主題相關論文，探討LLM和主題模型的研究趨勢與主題變化，做為未來研究學者發展方向之參考方向。

Modern artificial intelligence has significantly transformed human-computer interaction, particularly through the rise of generative interactive querying systems. This shift is driven by advancements in large language models, which excel at processing extensive textual data to identify keywords, research hotspots, and emerging trends, providing researchers with new academic exploration avenues. Topic models play a vital role by converting large volumes of text into coherent themes, facilitating content classification and keyword identification, thereby enhancing data retrieval efficiency. These models also dynamically track developments within research fields, offering insights into evolving directions and keyword popularity, reflecting the latest academic trends.
This study uses deep learning techniques to analyze the top ten most frequent keywords in large language models, examining their significance and how they reflect technological trends and shifts in research focus. By systematically analyzing LLM-related papers from the arXiv database (2007-2023), this research investigates the trends and evolution of topics related to LLMs and topic models, providing valuable insights for guiding future research directions.

論文審定書 i
誌謝 ii
摘要 iii
Abstract iv
目錄 v
圖次 vii
表次 viii
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機 2
1.3 研究目的 3
第二章 文獻探討 4
2.1 深度學習 DEEP LEARNING 4
2.2 大型語言模型 LARGE LANGUAGE MODEL 4
2.3 主題模型 TOPIC MODEL 5
2.4 非負矩陣 NONNEGATIVE MATRIX FACTORIZATION 6
2.5 DEEP NONNEGATIVE MATRIX FACTORIZATION 7
第三章 研究方法與步驟 8
3.1 研究方法 8
3.2 評估標準 12
第四章 實驗結果與分析 15
4.1 資料整理 15
4.2 研究流程 16
4.3 研究過程 17
4.4 研究分析 18
第五章 研究結論與建議 22
5.1 研究結論 22
5.2 研究建議 22
第六章 參考文獻 24
附錄 27

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners (arXiv:2005.14165). arXiv. http://arxiv.org/abs/2005.14165
Debain, Y. (n.d.). Deep Convolutional Nonnegative Autoencoders.
Elshamy, W. (2013). Continuous-time Infinite Dynamic Topic Models (arXiv:1302.7088). arXiv. http://arxiv.org/abs/1302.7088
Févotte, C., & Idier, J. (2011). Algorithms for nonnegative matrix factorization with the beta-divergence (arXiv:1010.1763). arXiv. http://arxiv.org/abs/1010.1763
Gillis, N. (2014). The Why and How of Nonnegative Matrix Factorization (arXiv:1401.5226). arXiv. http://arxiv.org/abs/1401.5226
Glynn, C., Tokdar, S. T., Howard, B., & Banks, D. L. (2019). Bayesian Analysis of Dynamic Linear Topic Models. Bayesian Analysis, 14(1). https://doi.org/10.1214/18-BA1100
Grootendorst, M. (2022). BERTopic: Neural topic modeling with a class-based TF-IDF procedure (arXiv:2203.05794). arXiv. http://arxiv.org/abs/2203.05794
Huang, S.-T., Kang, Y., Hung, S.-M., Kuo, B., & Cheng, I.-L. (2020). Topic Diffusion Discovery based on Deep Non-negative Autoencoder. 2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI), 405–408. https://doi.org/10.1109/IRI49571.2020.00067
Jelodar, H., Wang, Y., Yuan, C., Feng, X., Jiang, X., Li, Y., & Zhao, L. (2019). Latent Dirichlet allocation (LDA) and topic modeling: Models, applications, a survey. Multimedia Tools and Applications, 78(11), 15169–15211. https://doi.org/10.1007/s11042-018-6894-4
Kang, Y., Lin, K.-P., & Cheng, I.-L. (2018). Topic Diffusion Discovery Based on Sparseness-Constrained Non-Negative Matrix Factorization. 2018 IEEE International Conference on Information Reuse and Integration (IRI), 94–101. https://doi.org/10.1109/IRI.2018.00021
Kingma, D. P., & Welling, M. (2022). Auto-Encoding Variational Bayes (arXiv:1312.6114). arXiv. http://arxiv.org/abs/1312.6114
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788–791. https://doi.org/10.1038/44565
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2021). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv:2005.11401). arXiv. http://arxiv.org/abs/2005.11401
Liu, Y., He, H., Han, T., Zhang, X., Liu, M., Tian, J., Zhang, Y., Wang, J., Gao, X., Zhong, T., Pan, Y., Xu, S., Wu, Z., Liu, Z., Zhang, X., Zhang, S., Hu, X., Zhang, T., Qiang, N., … Ge, B. (2024). Understanding LLMs: A Comprehensive Overview from Training to Inference (arXiv:2401.02038). arXiv. http://arxiv.org/abs/2401.02038
Meng, Y., Zhang, Y., Huang, J., Zhang, Y., & Han, J. (2022). Topic Discovery via Latent Space Clustering of Pretrained Language Model Representations. Proceedings of the ACM Web Conference 2022, 3143–3152. https://doi.org/10.1145/3485447.3512034
Roberts, D. A., Yaida, S., & Hanin, B. (2022). The Principles of Deep Learning Theory. https://doi.org/10.1017/9781009023405
Saberi-Movahed, F., Berahman, K., Sheikhpour, R., Li, Y., & Pan, S. (2024). Nonnegative Matrix Factorization in Dimensionality Reduction: A Survey (arXiv:2405.03615). arXiv. http://arxiv.org/abs/2405.03615
Theis, L., Oord, A. van den, & Bethge, M. (2016). A note on the evaluation of generative models (arXiv:1511.01844). arXiv. http://arxiv.org/abs/1511.01844
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2023). Attention Is All You Need (arXiv:1706.03762). arXiv. http://arxiv.org/abs/1706.03762
Wan, M., Safavi, T., Jauhar, S. K., Kim, Y., Counts, S., Neville, J., Suri, S., Shah, C., White, R. W., Yang, L., Andersen, R., Buscher, G., Joshi, D., & Rangan, N. (2024). TnT-LLM: Text Mining at Scale with Large Language Models (arXiv:2403.12173). arXiv. http://arxiv.org/abs/2403.12173
Yang, X., Song, Z., King, I., & Xu, Z. (2023). A Survey on Deep Semi-supervised Learning. IEEE Transactions on Knowledge and Data Engineering, 35(9), 8934–8954. https://doi.org/10.1109/TKDE.2022.3220219
Zhao, H., Phung, D., Huynh, V., Jin, Y., Du, L., & Buntine, W. (2021). Topic Modelling Meets Deep Neural Networks: A Survey (arXiv:2103.00498). arXiv. http://arxiv.org/abs/2103.00498