在處理多標籤分類問題時，當前主流方法多為深度學習模型。然而，這些模型的決策過程對人類而言無法理解，因此難以提供合理的解釋來輔助人類做決策。為了解決這一問題，我們需要可解釋的模型，並且如果這些模型能夠模仿人類的決策方式將更為理想。為此，我們使用條件推論樹來產生容易理解的決策過程，並結合隨機森林演算法，以多棵樹的特性提升模型的表現。此外，我們融合了深度模型的架構，模仿深度學習的逐層學習方式，進而學習更複雜與多樣化的特徵。然而，由於多標籤分類問題的特性以及深度規則森林的複雜性，最終學到的規則往往非常繁瑣且冗長。為了解決這一問題，我們使用邏輯最佳化與規則選取的方法，篩選出最重要的規則，並探討不同特徵與規則之間的關係，幫助我們在現實生活中應用這些規則來做出更準確且可解釋的決策。

In addressing multi-label classification problems, deep learning models are the predominant methods currently in use. However, their decision-making processes are not comprehensible to humans, making it challenging to provide reasonable explanations for assisting human decision-making. Therefore, there is a need for interpretable models that can emulate human decision-making processes. To this end, we employ Conditional Inference Trees to generate easily understandable decision processes and enhance model performance through the ensemble approach of the Random Forest algorithm. Additionally, we integrate the structure of deep models to achieve layer-by-layer learning similar to deep learning, thereby capturing more complex and diverse features. However, due to the nature of multi-label classification and the complexity of the Deep Rule Forest, the resulting rules are often intricate and lengthy. To mitigate this, we employ logical optimization and rule selection methods to identify the most important rules and elucidate the relationships between different features and rules. This approach facilitates the application of these rules in real-world scenarios, enabling more accurate and interpretable decision-making.

論文審定書　i
摘要 ii
Abstract iii
List of Figures　v
List of Tables　vi
1. Introduction　1
2. Background　3
2.1 Multi-label Classification Task 3
2.2 Conditional Inference Tree　5
2.3 Representation Learning 7
2.4 Deep Architecture Models 9
2.5 Explainable Machine Learning and Interpretable Machine Learning　10
2.6 Logic Optimization (Rule Simplification) 12
3. Methodology 13
3.1 Deep Rule Forest 14
3.2 Conditional Inference Tree with Random Forest 16
3.3 Rule Extraction from Deep Rule Inference Forest 17
3.4 Rule Selection Using Lasso　20
4. Experiment 22
4.1 Experiment Setup 22
4.2 Model Performance Comparison 24
4.3 Interpretability with Learned Rules　27
5. Conclusion and Discussion 30
References　33

Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., ... & Herrera, F. (2020). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82-115. https://doi.org/10.1016/j.inffus.2019.12.012
Allaire, J., & Chollet, F. (2023). keras: R Interface to 'Keras'. R package version 2.6.1. https://cran.r-project.org/package=keras
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation Learning: A Review and New Perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828. https://doi.org/10.1109/TPAMI.2013.50
Blockeel, H., Džeroski, S., & Grbović, J. (2004). Simultaneous Prediction of Multiple Chemical Parameters of River Water Quality with TILDE. In Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery (PKDD), 32-40. https://doi.org/10.1007/978-3-540-48247-5_4
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth.
Charte, F., Charte, D., García, S., & Herrera, F. (2023). utiml: Utilities for Multi-Label Learning. R package version 0.1.3. https://cran.r-project.org/package=utiml
Charte, F., Rivera, A. J., del Jesus, M. J., & Herrera, F. (2015). Addressing imbalance in multilabel classification: Measures and random resampling algorithms. Neurocomputing, 163, 3-16. https://doi.org/10.1016/j.neucom.2014.08.091
Chekina, L., Gutfreund, D., Kontorovich, A., Rokach, L., & Shapira, B. (2013). Learning interpretable rules for multi-label classification. Machine Learning, 91(1), 1-42. https://doi.org/10.1007/s10994-013-5423-2
Clare, A., & King, R. D. (2001). Knowledge Discovery in Multi-label Phenotype Data. In L. De Raedt & A. Siebes (Eds.), Principles of Data Mining and Knowledge Discovery. PKDD 2001. Lecture Notes in Computer Science (Vol. 2168). Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44794-6_4
Elisseeff, A., & Weston, J. (2001). A Kernel Method for Multi-Labelled Classification. 14th Advances in Neural Information Processing Systems, 14, 681-687. https://doi.org/10.7551/mitpress/1120.003.0092
Esposito, F., Malerba, D., & Semeraro, G. (1995). Simplifying decision trees by pruning and grafting: New results (Extended abstract). In N. Lavrac & S. Wrobel (Eds.), Machine Learning: ECML-95. ECML 1995. Lecture Notes in Computer Science (Vol. 912). Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-59286-5_69
Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771-1800.
https://doi.org/10.1162/089976602760128018
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hothorn, T., Hornik, K., & Zeileis, A. (2006). Unbiased Recursive Partitioning: A Conditional Inference Framework. Journal of Computational and Graphical Statistics, 15(3), 651-674. https://doi.org/10.1198/106186006X133933
Hothorn, T., & Zeileis, A. (2015). partykit: A Modular Toolkit for Recursive Partytioning in R. Journal of Machine Learning Research, 16, 3905-3909. https://jmlr.org/papers/v16/hothorn15a.html
Ishwaran, H., & Kogalur, U. B. (2023). randomForestSRC: Fast Unified Random Forests for Survival, Regression, and Classification (RF-SRC). R package version 3.2.0. https://cran.r-project.org/package=randomForestSRC
Katakis, I., Tsoumakas, G., & Vlahavas, I. (2008). Multilabel Text Classification for Automated Tag Suggestion. In Proceedings of the ECML/PKDD 2008 Discovery Challenge. https://api.semanticscholar.org/CorpusID:15676013
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
Levshina, N. (2021). Conditional inference trees and random forests. In A practical handbook of corpus linguistics (pp. 611-643). Cham: Springer International Publishing.
Liu, H., Xu, X., & Li, J. J. (2017). A Bootstrap Lasso + Partial Ridge Method to Construct Confidence Intervals for Parameters in High-dimensional Sparse Linear Models. Statistica Sinica. https://doi.org/10.5705/ss.202018.0131
Molnar, C. (2019). Interpretable Machine Learning. Retrieved from https://christophm.github.io/interpretable-ml-book/
Pakrashi, A., Greene, D., & Mac Namee, B. (2016). Benchmarking Multi-label Classification Algorithms.
R Core Team. (2023). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
Rivolli, A., Parker, L., & de Carvalho, A. (2017). Food Truck Recommendation Using Multi-label Classification. In Proceedings of the International Conference on Intelligent Data Engineering and Automated Learning (IDEAL), 585-596. https://doi.org/10.1007/978-3-319-65340-2_48
RStudio Team. (2023). RStudio: Integrated Development Environment for R (Version 2023.3.0.386). RStudio, PBC. http://www.rstudio.com/
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288.
Trohidis, K., Tsoumakas, G., Kalliris, G., & Vlahavas, I. (2008). Multi-label classification of music into emotions. In Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR 2008) (pp. 325-330).
Turnbull, D., Barrington, L., Torres, D., & Lanckriet, G. (2008). Semantic Annotation and Retrieval of Music and Sound Effects. IEEE Transactions on Audio, Speech, and Language Processing, 16(3), 467-476.
https://doi.org/10.1109/TASL.2007.913750
Zhang, M.-L., & Zhou, Z.-H. (2005). A k-nearest neighbor based algorithm for multi-label classification. In 2005 IEEE International Conference on Granular Computing (Vol. 2, pp. 718-721). Beijing, China.
https://doi.org/10.1109/GRC.2005.1547385
Zhang, M.-L., & Zhou, Z.-H. (2006). Multilabel neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 18(10), 1338-1351. https://doi.org/10.1109/TKDE.2006.162
Zhang, M.-L., & Zhou, Z.-H. (2014). A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8), 1819-1837. https://doi.org/10.1109/TKDE.2013.39
Zhou, Z.-H., & Feng, J. (2017). Deep forest: Towards an alternative to deep neural networks. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. https://doi.org/10.24963/ijcai.2017/511
Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301-320. https://doi.org/10.1111/j.1467-9868.2005.00503.x