博碩士論文 etd-0806118-101202 詳細資訊


[回到前頁查詢結果 | 重新搜尋]

姓名 何讚容(Tsan-Jung He) 電子郵件信箱 E-mail 資料不公開
畢業系所 電機工程學系研究所(Electrical Engineering)
畢業學位 碩士(Master) 畢業時期 106學年第2學期
論文名稱(中) 基於RBF網路建立樣式分類之方法
論文名稱(英) A RBF network approach to pattern classification
檔案
  • etd-0806118-101202.pdf
  • 本電子全文僅授權使用者為學術研究之目的,進行個人非營利性質之檢索、閱讀、列印。
    請遵守中華民國著作權法之相關規定,切勿任意重製、散佈、改作、轉貼、播送,以免觸法。
    論文使用權限

    紙本論文:5 年後公開 (2023-09-07 公開)

    電子論文:使用者自訂權限:校內 5 年後、校外 5 年後公開

    論文語文/頁數 中文/53
    統計 本論文已被瀏覽 5635 次,被下載 0 次
    摘要(中) 長久以來分類模型都有廣泛的討論,而自動化分類模型更是在各種領域廣泛的運用。而放射狀基底函數網絡(Radial Basis Function network, 後稱RBF Network)在對於處理非線性系統有著良好的處理能力而廣泛的用來進行分類問題。但為了達到好的分類效果,建立RBF Network 模型會面臨許多參數的選擇,如決定隱藏層節點的個數、徑向基底函數參數設定以及最佳化參數值等問題。為了解決以上問題,我們提出一種新的RBF network的方法,對於解決隱藏層個數的問題,我們採用讓資料分布的狀況來決定隱藏層節點的個數,使用自適應分群演算法可以依照資料分布的特性來決定分群的群數,而RBF基底函數也從自適應分群演算法決定初始中心位置與標準差決定相似程度。我們也加入混合式的學習機制來達到網路的最佳化,包含最陡路徑倒傳遞法與最小平方法。我們的方法提供許多優勢,對於RBF網路中必須決定的隱藏層節點個數與徑向基底函數的參數都由訓練資料來決定,合理地建立模型的參數能夠提供一開始不錯的準確度,再經過最佳化參數的學習機制讓分類模型提升準確度。我們的方法能夠處理單一類別與多重類別的分類問題,並以多個實驗來顯示我們的效果。
    摘要(英) Radial basis function (RBF) networks are popularly applied to solving pattern classification problems. However, several issues are encountered in the applications, including the determination of the number of hidden nodes, the initial settings of the basis functions, and the refinement of the parameter values. In this paper, we propose a novel RBF network approach for classification applications. An iterative self-constructing clustering algorithm is used to determine the number of hidden nodes in the hidden layer. Basis functions are generated, and their centers and deviations are initialized according to the data distribution of the formed clusters. To learn optimal values of the network parameters, a hybrid learning strategy, involving steepest descent back-propagation and least squares method, is adopted. Hyperbolic tangent sigmoid functions and Tikhonov regularization are employed. As a result, optimized RBF networks are obtained. Our approach can offer advantages in practicality. The number of hidden nodes and initial settings of the basis functions are determined automatically. Through the incorporation of adaptive deviations, data can be described more appropriately by the basis functions. The adopted optimization learning allows more accurate predictions to be made. Furthermore, our approach is applicable to both single-label and multi-label classification problems. A number of experiments are conducted to show the effectiveness of the proposed approach.
    關鍵字(中)
  • 支援向量機
  • 極限學習機
  • 自適應遞增學習
  • 監督式學習
  • 網路攻擊
  • 關鍵字(英)
  • Pattern classification
  • Self-constructing clustering
  • Basis functions
  • Steepest descent
  • Back-propagation algorithm
  • 論文目次 論文審定書. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
    論文公開授權書. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
    致謝. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
    中文摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
    英文摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
    目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
    圖目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
    表目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
    第一章緒論. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
    1.1 研究背景. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
    1.2 研究動機. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
    1.3 方法與論文架構. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
    第二章文獻探討. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
    第三章研究方法. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
    3.1 樣式分類問題. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
    3.2 方法架構. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
    3.2.1 階段一、建構網路模型. . . . . . . . . . . . . . . . . . . . . . 10
    3.2.2 階段二、參數最佳化. . . . . . . . . . . . . . . . . . . . . . . 17
    3.2.3 方法總結. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
    3.3 實例說明. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
    第四章實驗與結果分析. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
    4.1 評估指標. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
    4.2 資料集. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
    4.3 實驗一. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
    4.4 實驗二. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
    4.5 實驗三. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
    第五章結論. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
    參考文獻. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
    參考文獻 [1] H. Leung, T. Lo, and S. Wang, “Prediction of noisy chaotic time series using an optimal
    radial basis function neural network,” IEEE Transactions on Neural Networks,
    vol. 12, no. 5, pp. 1163–1172, Sep 2001.
    [2] B. S. Lin, B. S. Lin, F. C. Chong, and F. Lai, “Higher-order-statistics-based radial
    basis function networks for signal enhancement,” IEEE Transactions on Neural Networks,
    vol. 18, no. 3, pp. 823–832, May 2007.
    [3] I. Maglogiannis, H. Sarimveis, C. T. Kiranoudis, A. A. Chatziioannou,
    N. Oikonomou, and V. Aidinis, “Radial basis function neural networks classification
    for the recognition of idiopathic pulmonary fibrosis in microscopic images,”
    IEEE Transactions on Information Technology in Biomedicine, vol. 12, no. 1, pp.
    42–54, Jan 2008.
    [4] A. Kusiak, H. Zheng, and Z. Song, “Short-term prediction of wind farm power: A
    data mining approach,” IEEE Transactions on Energy Conversion, vol. 24, no. 1, pp.
    125–136, March 2009.
    [5] S. M. Chen and C. D. Chen, “Taiex forecasting based on fuzzy time series and fuzzy
    variation groups,” IEEE Transactions on Fuzzy Systems, vol. 19, no. 1, pp. 1–12,
    Feb 2011.
    [6] S. M. Chen, H. P. Chu, and T. W. Sheu, “Taiex forecasting using fuzzy time series and
    automatically generated weights of multiple factors,” IEEE Transactions on Systems,
    Man, and Cybernetics - Part A: Systems and Humans, vol. 42, no. 6, pp. 1485–1495,
    Nov 2012.
    [7] D. Zissis, E. K. Xidias, and D. Lekkas, “A cloud based architecture capable of
    perceiving and predicting multiple vessel behaviour,” Applied Soft Computing,
    36
    vol. 35, pp. 652 – 661, 2015. [Online]. Available: http://www.sciencedirect.com/
    science/article/pii/S1568494615004329
    [8] R. Ak, O. Fink, and E. Zio, “Two machine learning approaches for short-term wind
    speed time-series prediction,” IEEE Transactions on Neural Networks and Learning
    Systems, vol. 27, no. 8, pp. 1734–1747, Aug 2016.
    [9] M. Ozay, I. Esnaola, F. T. Y. Vural, S. R. Kulkarni, and H. V. Poor, “Machine learning
    methods for attack detection in the smart grid,” IEEE Transactions on Neural
    Networks and Learning Systems, vol. 27, no. 8, pp. 1773–1786, Aug 2016.
    [10] R. G. Sutar and A. G. Kothari, “Intelligent electrocardiogram pattern classification
    and recognition using low-cost cardio-care system,” IET Science, Measurement
    Technology, vol. 9, no. 1, pp. 134–143, 2015.
    [11] J. P. R. R. Leite and R. L. Moreno, “Heartbeat classification with low computational
    cost using hjorth parameters,” IET Signal Processing, vol. 12, no. 4, pp. 431–438,
    2018.
    [12] S. Haykin, Neural Networks: A Comprehensive Foundation, 1st ed. Upper Saddle
    River, NJ, USA: Prentice Hall PTR, 1994.
    [13] J. R. Quinlan, “Simplifying decision trees,” International Journal of Human-
    Computer Studies, vol. 51, no. 2, pp. 497–510, 1999.
    [14] D. Y. Ignatov and A. D. Ignatov, “Decision stream: Cultivating deep decision trees,”
    in Proceedings of IEEE 29th International Conference on Tools for Artificial Intelligence,
    2017, pp. 905–912.
    [15] D. W. Aha, Editorial. Dordrecht: Springer Netherlands, 1997, pp. 7–10.
    [16] S. J. Russell and P. Norvig, Artificial intelligence: a modern approach. Malaysia;
    Pearson Education Limited„ 2016.
    37
    [17] C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20,
    no. 3, pp. 273–297, Sep 1995. [Online]. Available: https://doi.org/10.1007/
    BF00994018
    [18] J. Suykens and J. Vandewalle, “Least squares support vector machine classifiers,”
    Neural Processing Letters, vol. 9, no. 3, pp. 293–300, Jun 1999. [Online]. Available:
    https://doi.org/10.1023/A:1018628609742
    [19] W. F. Schmidt, M. A. Kraaijveld, and R. P. W. Duin, “Feedforward neural networks
    with random weights,” in Proceedings of 11th IAPR International Conference on
    Pattern Recognition, Conference B: Pattern Recognition Methodology and Systems,
    vol. II, June 1992, pp. 1–4.
    [20] Y. Pao, G. H. Park, and D. J. Sobajic, “Learning and generalization characteristics
    of random vector functional-link net,” Neurocomputing, vol. 6, pp. 163–180, 1994.
    [21] B. Tang and H. He, “Enn: Extended nearest neighbor method for pattern recognition
    [research frontier],” IEEE Computational Intelligence Magazine, vol. 10, no. 3, pp.
    52–60, Aug 2015.
    [22] J. Feng, Y. Wei, and Q. Zhu, “Natural neighborhood-based classification algorithm
    without parameter k,” Big Data Mining and Analytics, vol. 1, no. 4, pp. 257–265,
    December 2018.
    [23] Z. G. Liu, Q. Pan, G. Mercier, and J. Dezert, “A new incomplete pattern classification
    method based on evidential reasoning,” IEEE Transactions on Cybernetics, vol. 45,
    no. 4, pp. 635–646, April 2015.
    [24] S. Z. Dadaneh, E. R. Dougherty, and X. Qian, “Optimal bayesian classification with
    missing values,” IEEE Transactions on Signal Processing, vol. 66, no. 16, pp. 4182–
    4192, Aug 2018.
    38
    [25] Z. Liu, C. Zhuo, and X. Xu, “Efficient segmentation method using quantised and
    non-linear cenn for breast tumour classification,” Electronics Letters, vol. 54, no. 12,
    pp. 737–738, 2018.
    [26] D. Dey, B. Chatterjee, S. Dalai, S. Munshi, and S. Chakravorti, “A deep learning
    framework using convolution neural network for classification of impulse fault patterns
    in transformers with increased accuracy,” IEEE Transactions on Dielectrics
    and Electrical Insulation, vol. 24, no. 6, pp. 3894–3897, Dec 2017.
    [27] M. U. Asad, U. Farooq, J. Gu, J. Amin, A. Sadaqat, M. E. El-Hawary, and J. Luo,
    “Neo-fuzzy supported brain emotional learning based pattern recognizer for classification
    problems,” IEEE Access, vol. 5, pp. 6951–6968, 2017.
    [28] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: a new learning
    scheme of feedforward neural networks,” in Proceedings of 2004 IEEE International
    Joint Conference on Neural Networks (IEEE Cat. No.04CH37541), vol. 2, July 2004,
    pp. 985–990 vol.2.
    [29] G. B. Huang, H. Zhou, X. Ding, and R. Zhang, “Extreme learning machine for regression
    and multiclass classification,” IEEE Transactions on Systems, Man, and
    Cybernetics, Part B (Cybernetics), vol. 42, no. 2, pp. 513–529, April 2012.
    [30] H. Wen, H. Fan, W. Xie, and J. Pei, “Hybrid structure-adaptive rbf-elm network
    classifier,” IEEE Access, vol. 5, pp. 16 539–16 554, 2017.
    [31] M. J. Orr et al., “Introduction to radial basis function networks,” 1996.
    [32] A. Ferreira and M. Figueiredo, “Hybrid generative/discriminative training of radial
    basis function networks.” Proceedings of European Symposium on Artificial Neural
    Networks, pp. 599–604, Apr 2006.
    39
    [33] M.-W. Mak and S.-Y. Kung, “Estimation of elliptical basis function parameters by
    the em algorithm with application to speaker verification,” IEEE Transactions on
    Neural Networks, vol. 11, no. 4, pp. 961–969, Jul 2000.
    [34] J.-C. Luo, Y. Leung, J. Zheng, and J.-H. Ma, “An elliptical basis function
    network for classification of remote sensing images,” Journal of Geographical
    Systems, vol. 6, no. 3, pp. 219–236, Oct 2004. [Online]. Available: https:
    //doi.org/10.1007/s10109-004-0136-1
    [35] S. Jaiyen, C. Lursinsap, and S. Phimoltares, “A very fast neural learning for classification
    using only new incoming datum,” IEEE Transactions on Neural Networks,
    vol. 21, no. 3, pp. 381–392, March 2010.
    [36] M. K. Titsias and A. C. Likas, “Shared kernel models for class conditional density
    estimation,” IEEE Transactions on Neural Networks, vol. 12, no. 5, pp. 987–997,
    Sep 2001.
    [37] X. Yang, Y. Li, Y. Sun, T. Long, and T. K. Sarkar, “Fast and robust rbf neural network
    based on global k-means clustering with adaptive selection radius for sound source
    angle estimation,” IEEE Transactions on Antennas and Propagation, vol. 66, no. 6,
    pp. 3097–3107, June 2018.
    [38] Y. Freund and R. E. Schapire, “A decision-theoretic generalization of online
    learning and an application to boosting,” Journal of Computer and
    System Sciences, vol. 55, no. 1, pp. 119 – 139, 1997. [Online]. Available:
    http://www.sciencedirect.com/science/article/pii/S002200009791504X
    [39] M. Lapin, M. Hein, and B. Schiele, “Analysis and optimization of loss functions
    for multiclass, top-k, and multilabel classification,” IEEE Transactions on Pattern
    Analysis and Machine Intelligence, vol. 40, no. 7, pp. 1533–1554, July 2018.
    40
    [40] J. Nam, J. Kim, E. Loza Mencía, I. Gurevych, and J. Fürnkranz, “Large-scale multilabel
    text classification — revisiting neural networks,” in Proceedings of European
    Conference on Machine Learning and Knowledge Discovery in Databases,
    T. Calders, F. Esposito, E. Hüllermeier, and R. Meo, Eds. Berlin, Heidelberg:
    Springer Berlin Heidelberg, 2014, pp. 437–452.
    [41] G. Tsoumakas and I. Katakis, “Multi-label classification: An overview,” International
    Journal of Data Warehousing and Mining (IJDWM), vol. 3, no. 3, pp. 1–13,
    2007.
    [42] G. Madjarov, D. Kocev, D. Gjorgjevikj, and S. Džeroski, “An extensive
    experimental comparison of methods for multi-label learning,” Pattern Recognition,
    vol. 45, no. 9, pp. 3084 – 3104, 2012, best Papers of Iberian Conference
    on Pattern Recognition and Image Analysis (IbPRIA’2011). [Online]. Available:
    http://www.sciencedirect.com/science/article/pii/S0031320312001203
    [43] J. Read, B. Pfahringer, G. Holmes, and E. Frank, “Classifier chains for multi-label
    classification,” Machine Learning, vol. 85, no. 3, p. 333, Jun 2011. [Online].
    Available: https://doi.org/10.1007/s10994-011-5256-5
    [44] K. Trohidis, G. Tsoumakas, G. Kalliris, and I. Vlahavas, “Multi-label classification
    of music by emotion,” EURASIP Journal on Audio, Speech, and Music
    Processing, vol. 2011, no. 1, p. 4, Sep 2011. [Online]. Available: https:
    //doi.org/10.1186/1687-4722-2011-426793
    [45] M. R. Boutell, J. Luo, X. Shen, and C. M. Brown, “Learning multi-label scene
    classification,” Pattern Recognition, vol. 37, no. 9, pp. 1757 – 1771, 2004. [Online].
    Available: http://www.sciencedirect.com/science/article/pii/S0031320304001074
    [46] G. Tsoumakas, E. Spyromitros-Xioufis, J. Vilcek, and I. Vlahavas, “Mulan: A java
    41
    library for multi-label learning,” Journal of Machine Learning Research, vol. 12, no.
    Jul, pp. 2411–2414, 2011.
    [47] G. Tsoumakas, I. Katakis, and I. Vlahavas, “Random k-labelsets for multilabel classification,”
    IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 7,
    pp. 1079–1089, July 2011.
    [48] H. Y. Lo, S. D. Lin, and H. M. Wang, “Generalized k-labelsets ensemble for multilabel
    and cost-sensitive classification,” IEEE Transactions on Knowledge and Data
    Engineering, vol. 26, no. 7, pp. 1679–1691, July 2014.
    [49] M. L. Zhang and Z. H. Zhou, “A review on multi-label learning algorithms,” IEEE
    Transactions on Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819–1837,
    Aug 2014.
    [50] H. Li, Y. jian Guo, M. Wu, P. Li, and Y. Xiang, “Combine multi-valued attribute
    decomposition with multi-label learning,” Expert Systems with Applications, vol. 37,
    no. 12, pp. 8721 – 8728, 2010. [Online]. Available: http://www.sciencedirect.com/
    science/article/pii/S0957417410005488
    [51] M.-L. Zhang and Z.-H. Zhou, “Multilabel neural networks with applications to functional
    genomics and text categorization,” IEEE Transactions on Knowledge and
    Data Engineering, vol. 18, no. 10, pp. 1338–1351, Oct 2006.
    [52] M.-L. Zhang and Z.-H. Zhou, “ML-KNN: A lazy learning approach to multi-label
    learning,” Pattern recognition, vol. 40, no. 7, pp. 2038–2048, 2007.
    [53] M.-L. Zhang, “Ml-rbf: Rbf neural networks for multi-label learning,” Neural
    Processing Letters, vol. 29, no. 2, pp. 61–74, Apr 2009. [Online]. Available:
    https://doi.org/10.1007/s11063-009-9095-3
    42
    [54] S. J. Lee and J. Y. Jiang, “Multilabel text categorization based on fuzzy relevance
    clustering,” IEEE Transactions on Fuzzy Systems, vol. 22, no. 6, pp. 1457–1471,
    Dec 2014.
    [55] G. Kurata, B. Xiang, and B. Zhou, “Improved neural network-based multi-label classification
    with better initialization leveraging label co-occurrence,” in Proceedings
    of the 2016 Conference of the North American Chapter of the Association for Computational
    Linguistics: Human Language Technologies, 2016, pp. 521–526.
    [56] D. H. Broomhead and D. Lowe, “Multivariable functional interpolation and adaptive
    networks,” Complex Systems, vol. 2, pp. 321–355, 1988.
    [57] G. H. Golub and C. F. Van Loan, Matrix computations. JHU Press, 2012, vol. 3.
    [58] B. Widrow and R. Winter, “Neural nets for adaptive filtering and adaptive pattern
    recognition,” IEEE Computer Magazine, vol. 21, no. 3, pp. 25–39, 1988.
    [59] M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for
    classification tasks,” Information Processing and Management, vol. 45, pp. 427–
    437, 2009.
    [60] A. Asuncion and D. Newman, “UCI machine learning repository,” 2007.
    [61] R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, “LIBLINEAR: A
    library for large linear classification,” Journal of Machine Learning Research, vol. 9,
    pp. 1871–1874, 2008.
    [62] A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,” 2004.
    [63] C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM
    Transactions on Intelligent Systems and Technology, vol. 2, pp. 27:1–27:27, 2011,
    software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
    43
    [64] N. M. Nawi, M. R. Ransing, and R. S. Ransing, “An improved learning algorithm
    based on the broyden-fletcher-goldfarb-shanno (bfgs) method for back propagation
    neural networks,” in Sixth International Conference on Intelligent Systems Design
    and Applications, vol. 1, Oct 2006, pp. 152–157.
    [65] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel,
    P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau,
    M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in
    Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.
    [66] L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae,
    P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt,
    and G. Varoquaux, “API design for machine learning software: experiences from the
    scikit-learn project,” in ECML PKDD Workshop: Languages for Data Mining and
    Machine Learning, 2013, pp. 108–122.
    [67] G. E. Hinton, “Connectionist learning procedures,” in Machine Learning, Volume
    III. Elsevier, 1990, pp. 555–610.
    [68] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward
    neural networks,” in Proceedings of the thirteenth international conference on
    artificial intelligence and statistics, 2010, pp. 249–256.
    口試委員
  • 吳志宏 - 召集委員
  • 侯俊良 - 委員
  • 劉志峰 - 委員
  • 歐陽振森 - 委員
  • 李錫智 - 指導教授
  • 口試日期 2018-07-26 繳交日期 2018-09-07

    [回到前頁查詢結果 | 重新搜尋]


    如有任何問題請與論文審查小組聯繫