近年來，終身學習(Continual Learning, CL)[29, 39]已成為深度學習領域中受到廣泛關注的研究方向，其目標是訓練出能夠處理多個不同任務的模型，同時維持模型在所有已學習任務上之預測性能。然而，神經網路固有的穩定性-可塑性困境[15]使終身學習技術面臨災難性遺忘(Catastrophic Forgetting, CF)與跨任務之知識轉移(Knowledge Transfer, KT)兩大挑戰，許多先前研究也為了解決上述挑戰而發展出不同的系列方法與分支。
目前，CL技術仍主要應用於電腦視覺領域，在自然語言處理(Natural Language Processing, NLP)領域當中存有許多發展空間，近期有部分研究逐漸將CL方法應用於文字分類與對話生成等NLP任務中，但有關多領域(Multi-Domain)對話推薦任務方面的研究仍非常稀少，又隨著社會對於個人資料保護的意識逐年上升，以及高昂的運算成本，能夠同時學習多個任務的Multi-Task Learning(MTL)方法也將在未來實務中受到限制，因此，將CL技術融入對話推薦領域是一個極具發展潛力的應用。
綜上所述，本研究提出了基於終身學習的Unfrozen CTR+Adaptive EWC方法，並將其應用於多領域對話推薦系統當中，目的在於將CL技術與對話推薦系統當中的項目推薦模組進行結合，使推薦系統能夠預測不同領域的推薦任務。而為減輕終身學習特有的災難性遺忘與知識轉移問題，本研究同時透過在BERT模型架構中加入Ke, Zixuan [20]學者等人所提出之CL-Plugin插件，以及在訓練過程中採用基於EWC(Elastic Weight Consolidation, EWC)進行改善的自適應正規化方法，使多領域對話推薦系統在所有領域任務上擁有穩定的預測性能。
本研究在MultiWOZ-2.2與REDIAL資料集上進行完整的實驗，證實了在對話推薦模型中同樣會出現遺忘現象，並探討了在不同任務序列下，相似任務與不相似任務對於模型預測之影響，最後，透過實驗分析驗證了Unfrozen CTR+Adaptive EWC方法在持續學習新的領域任務上具備有效性與廣適性，且與被視為性能上限的MTL方法相較之下，本研究方法也表現出與其相當的競爭力。

In recent years, Continual Learning (CL)[29, 39] has become a widely studied research direction in deep learning. Its goal is to train models capable of handling multiple distinct tasks while maintaining satisfactory performance on all previously learned tasks. However, the inherent stability-plasticity dilemma[15] of neural networks poses significant challenges for CL techniques, including Catastrophic Forgetting (CF) and Knowledge Transfer (KT) across tasks. Previous research has proposed various methods and branches to address these challenges.
Currently, CL techniques are primarily applied in computer vision tasks, with numerous natural language processing (NLP) developments remaining unexplored. Although some studies have started to adopt CL methods in tasks like text classification and dialogue generation, research on multi-domain conversational recommendation tasks remains scarce. With increasing awareness of personal data protection and the high computational costs, the feasibility of employing Multi-Task Learning (MTL) methods to handle multiple tasks simultaneously may be limited. Therefore, integrating CL techniques into conversational recommendation systems is a promising application area.
In this study, we propose the CL-based Unfrozen CTR+Adaptive EWC approach and apply it to multi-domain conversational recommendation systems. We aim to combine CL techniques with the item recommendation module of the conversational recommendation system, enabling the system to handle diverse recommendation tasks across different domains. To alleviate the challenges of CF and KT in CL, we integrate the CL-Plugin proposed by Ke, Zixuan [20], into the BERT model architecture and apply the Elastic Weight Consolidation (EWC) technique in an improved adaptive regularization method during training. These efforts ensure stable predictive performance on all domain tasks for the multi-domain conversational recommendation system.
Through comprehensive experiments on the MultiWOZ-2.2 and REDIAL datasets, we confirm the forgetting phenomenon in dialogue recommendation models and investigate the impact of task similarity and dissimilarity on predictive performance under different task sequences. Furthermore, we verify the effectiveness and versatility of the Unfrozen CTR+Adaptive EWC method in continual learning of new domain tasks, showing competitive performance compared to the MTL methods.

論文審定書 i
誌謝 ii
摘要 iii
Abstract iv
目錄 vi
圖次 ix
表次 xi
第一章 緒論 1
1.1 研究背景 1
1.2 研究動機 1
1.2.1 終身學習於自然語言處理之應用 1
1.2.2 多領域對話推薦 2
1.3 研究目的 3
第二章 文獻探討 4
2.1 終身學習(Continual Learning, CL) 4
2.1.1 Task-Incremental Learning vs. Class-Incremental Learning 4
2.1.2 災難性遺忘(Catastrophic Forgetting, CF) 5
2.1.3 知識轉移(Knowledge Transfer, KT) 5
2.2 終身學習系列方法與分支 5
2.2.1 回放(Replay) 6
2.2.2 正規化(Regularization-based) 6
2.2.3 參數隔離(Parameter Isolation) 8
2.3 終身學習於自然語言處理研究之應用 9
2.3.1 CTR—基於意見目標之情感分類與文字分類 9
2.3.2 TPEM—基於任務導向之對話系統 11
2.3.3 ARPER—自然語言生成 12
2.4 深度對話推薦系統(Deep Conversational Recommendation System) 13
2.4.1 TG-ReDial 14
2.4.2 Deep Convolutional Recommender (DCR) 15
2.5 預訓練模型(Pre-Trained Model, PTM) 17
第三章 研究方法與步驟 19
3.1 模型架構 19
3.2 Unfrozen CTR+Adaptive EWC 20
3.2.1 Unfrozen CTR 20
3.2.2 Adaptive Elastic Weight Consolidation (A-EWC) 21
第四章 實驗結果與討論 27
4.1 資料集介紹 27
4.1.1 MultiWOZ-2.2 27
4.1.2 REDIAL 28
4.2 基線比較(Compared Baselines) 28
4.3 實驗流程與設計 29
4.3.1 Three tasks of MultiWOZ-2.2 30
4.3.2 Six tasks of MultiWOZ-2.2 31
4.3.3 Four tasks of MultiWOZ-2.2 and REDIAL 32
4.4 評估方式 33
4.5 實驗結果與分析 34
4.5.1 任務相似度之影響 34
4.5.2 CRS 模型的遺忘現象 36
4.5.3 整體結果與討論 39
4.5.4 消融研究 (Ablation Study) 40
4.5.5 訓練成本分析 41
4.5.6 相似度估計方式之探討 42
第五章 結論與未來展望 45
5.1 結論 45
5.2 未來展望 45
參考文獻 47

[1] A. Aich, "Elastic weight consolidation (EWC): Nuts and bolts," arXiv preprint arXiv:2105.04093, 2021.
[2] R. Aljundi, F. Babiloni, M. Elhoseiny, M. Rohrbach, and T. Tuytelaars, "Memory aware synapses: Learning what (not) to forget," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 139-154.
[3] R. Aljundi, P. Chakravarty, and T. Tuytelaars, "Expert gate: Lifelong learning with a network of experts," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3366-3375.
[4] P. P. Brahma and A. Othon, "Subset replay based continual learning for scalable improvement of autonomous systems," in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018: IEEE, pp. 1179-11798.
[5] P. Budzianowski et al., "MultiWOZ--a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling," arXiv preprint arXiv:1810.00278, 2018.
[6] G. Castellucci, S. Filice, D. Croce, and R. Basili, "Learning to solve NLP tasks in an incremental number of languages," in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2021, pp. 837-847.
[7] A. Chaudhry, P. K. Dokania, T. Ajanthan, and P. H. Torr, "Riemannian walk for incremental learning: Understanding forgetting and intransigence," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 532-547.
[8] Z. Chen and B. Liu, "Lifelong machine learning," Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 12, no. 3, pp. 1-207, 2018.
[9] M. De Lange et al., "A continual learning survey: Defying forgetting in classification tasks," IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 7, pp. 3366-3385, 2021.
[10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
[11] K. Doshi and Y. Yilmaz, "Continual learning for anomaly detection in surveillance videos," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2020, pp. 254-255.
[12] B. Ermis, G. Zappella, M. Wistuba, and C. Archambeau, "Memory efficient continual learning for neural text classification," arXiv preprint arXiv:2203.04640, 2022.
[13] H. Fang, C. Chen, Y. Long, G. Xu, and Y. Xiao, "DTCRSKG: A Deep Travel Conversational Recommender System Incorporating Knowledge Graph," Mathematics, vol. 10, no. 9, p. 1402, 2022.
[14] B. Geng, F. Yuan, Q. Xu, Y. Shen, R. Xu, and M. Yang, "Continual learning for task-oriented dialogue system with iterative network pruning, expanding and masking," arXiv preprint arXiv:2107.08173, 2021.
[15] S. T. Grossberg, Studies of mind and brain: Neural principles of learning, perception, development, cognition, and motor control. Springer Science & Business Media, 2012.
[16] C.-Y. Hung, C.-H. Tu, C.-E. Wu, C.-H. Chen, Y.-M. Chan, and C.-S. Chen, "Compacting, picking and growing for unforgetting continual learning," Advances in Neural Information Processing Systems, vol. 32, 2019.
[17] S. C. Hung, J.-H. Lee, T. S. Wan, C.-H. Chen, Y.-M. Chan, and C.-S. Chen, "Increasingly packing multiple facial-informatics modules in a unified deep-learning model via lifelong learning," in Proceedings of the 2019 on International Conference on Multimedia Retrieval, 2019, pp. 339-343.
[18] W.-C. Kang and J. McAuley, "Self-attentive sequential recommendation," in 2018 IEEE international conference on data mining (ICDM), 2018: IEEE, pp. 197-206.
[19] Z. Ke, B. Liu, and X. Huang, "Continual learning of a mixed sequence of similar and dissimilar tasks," Advances in Neural Information Processing Systems, vol. 33, pp. 18493-18504, 2020.
[20] Z. Ke, B. Liu, N. Ma, H. Xu, and L. Shu, "Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning," Advances in Neural Information Processing Systems, vol. 34, pp. 22443-22456, 2021.
[21] J. Kirkpatrick et al., "Overcoming catastrophic forgetting in neural networks," Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521-3526, 2017.
[22] S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang, "Overcoming catastrophic forgetting by incremental moment matching," Advances in neural information processing systems, vol. 30, 2017.
[23] R. Li, S. Ebrahimi Kahou, H. Schulz, V. Michalski, L. Charlin, and C. Pal, "Towards deep conversational recommendations," Advances in neural information processing systems, vol. 31, 2018.
[24] L. Liao, R. Takanobu, Y. Ma, X. Yang, M. Huang, and T.-S. Chua, "Deep conversational recommender in travel," arXiv preprint arXiv:1907.00710, 2019.
[25] D. Lopez-Paz and M. A. Ranzato, "Gradient episodic memory for continual learning," Advances in neural information processing systems, vol. 30, 2017.
[26] A. Mallya and S. Lazebnik, "Packnet: Adding multiple tasks to a single network by iterative pruning," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 7765-7773.
[27] M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, "Class-incremental learning: survey and performance evaluation on image classification," arXiv preprint arXiv:2010.15277, 2020.
[28] F. Mi, L. Chen, M. Zhao, M. Huang, and B. Faltings, "Continual learning for natural language generation in task-oriented dialog systems," arXiv preprint arXiv:2010.00910, 2020.
[29] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, "Continual lifelong learning with neural networks: A review," Neural Networks, vol. 113, pp. 54-71, 2019.
[30] J. Pfeiffer, A. Kamath, A. Rücklé, K. Cho, and I. Gurevych, "AdapterFusion: Non-destructive task composition for transfer learning," arXiv preprint arXiv:2005.00247, 2020.
[31] X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, "Pre-trained models for natural language processing: A survey," Science China Technological Sciences, vol. 63, no. 10, pp. 1872-1897, 2020.
[32] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language models are unsupervised multitask learners," OpenAI blog, vol. 1, no. 8, p. 9, 2019.
[33] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, "icarl: Incremental classifier and representation learning," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2017, pp. 2001-2010.
[34] M. Riemer, T. Klinger, D. Bouneffouf, and M. Franceschini, "Scalable recollections for continual lifelong learning," in Proceedings of the AAAI Conference on Artificial Intelligence, 2019, vol. 33, no. 01, pp. 1352-1359.
[35] A. A. Rusu et al., "Progressive neural networks," arXiv preprint arXiv:1606.04671, 2016.
[36] J. Schwarz et al., "Progress & compress: A scalable framework for continual learning," in International Conference on Machine Learning, 2018: PMLR, pp. 4528-4537.
[37] G. Shan, S. Xu, L. Yang, S. Jia, and Y. Xiang, "Learn#: a novel incremental learning method for text classification," Expert Systems with Applications, vol. 147, p. 113198, 2020.
[38] H. Shin, J. K. Lee, J. Kim, and J. Kim, "Continual learning with deep generative replay," Advances in neural information processing systems, vol. 30, 2017.
[39] S. Thrun, "A lifelong learning perspective for mobile robot control," in Intelligent robots and systems, 1995: Elsevier, pp. 201-214.
[40] D. H. Tran et al., "Deep Conversational Recommender Systems: A New Frontier for Goal-Oriented Dialogue Systems," arXiv preprint arXiv:2004.13245, 2020.
[41] C.-S. Wu, R. Socher, and C. Xiong, "Global-to-local memory pointer networks for task-oriented dialogue," arXiv preprint arXiv:1901.04713, 2019.
[42] X. Zang, A. Rastogi, S. Sunkara, R. Gupta, J. Zhang, and J. Chen, "Multiwoz 2.2: A dialogue dataset with additional annotation corrections and state tracking baselines," arXiv preprint arXiv:2007.12720, 2020.
[43] F. Zenke, B. Poole, and S. Ganguli, "Continual learning through synaptic intelligence," in International Conference on Machine Learning, 2017: PMLR, pp. 3987-3995.
[44] M. Zhai, L. Chen, F. Tung, J. He, M. Nawhal, and G. Mori, "Lifelong gan: Continual learning for conditional image generation," in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2759-2768.
[45] S. Zhang, L. Yao, A. Sun, and Y. Tay, "Deep learning based recommender system: A survey and new perspectives," ACM Computing Surveys (CSUR), vol. 52, no. 1, pp. 1-38, 2019.
[46] K. Zhou, Y. Zhou, W. X. Zhao, X. Wang, and J.-R. Wen, "Towards topic-guided conversational recommender system," arXiv preprint arXiv:2010.04125, 2020.