世界各國的交通事故中，因疲勞而引發的事故比例相當高，駕駛車輛需要高度的精神集中以注意道路上的任何情況，因此，若能夠利用電腦視覺技術協助判斷駕駛人的疲勞狀態，將有助於減少此類事故的發生，避免許多悲劇事件的發生。在過去駕駛疲勞偵測的資料集主要以西方歐美人士為主，為了建立更符合東方人眼睛特徵的資料集，我們自行創建了一個有多種情境，包含8位男性和3位女性，總共有34支影片的資料集，影片內容有配戴口罩、配戴近視眼鏡、以及同時配戴口罩和近視眼鏡的情境，其中也涵蓋東方人常見的小眼睛特徵於本研究資料集中，這些特點都與目前常見的駕駛疲勞資料集有所不同，本研究還開發了一個基於深度學習的疲勞檢測系統用於疲勞檢測使用，在辨別疲勞與否時的一個常見定義，是判斷在特定時間內是否長時間閉眼，當給定一個影片時，我們首先使用MTCNN來偵測每一幀影像中人臉的位置，然後使用卷積神經網路來判斷偵測到的人臉的眼睛是否開啟或閉合，最後再根據我們自行建立的資料集進行測試，結果顯示，在開眼與閉眼的辨識方面，雖然臉部配戴口罩或眼鏡配件會對模型的表現產生影響，但我們的模型仍然達到了92% 的準確度，而在疲勞判斷方面，我們使用了15% 和30% 的PERCLOS兩種疲勞條件來進行判斷，在一般人臉的情況下，模型達到了98% 的準確度，此外，我們還使用了眼部資料集進行疲勞測試，發現在具有口罩的測試影片中，由於人臉偵測的眼睛特徵位置偶爾會偏移，這導致眼部無法準確擷取，從而影響了模型的判斷效果。

In traffic accidents worldwide, fatigue-induced incidents comprise a significantly high proportion. Operating a vehicle demands a high level of mental focus to remain attentive to any conditions on the road. Therefore, if computer vision technology can be utilized to assist in assessing the fatigue state of drivers, it could contribute to reducing the occurrence of such accidents and preventing many tragic events. Historically, driver fatigue detection datasets have primarily centered on Western individuals. To establish a dataset more aligned with the eye characteristics of Eastern individuals, we independently created a dataset with multiple scenarios. This dataset includes 8 males and 3 females, totaling 34 video clips. The video content encompasses scenarios involving individuals wearing masks, wearing prescription glasses, and simultaneously wearing both masks and prescription glasses. Additionally, it incorporates common eye features of Eastern individuals, setting it apart from currently prevalent driver fatigue datasets. In this study, we also developed a deep learning-based fatigue detection system for fatigue assessment. A common criterion for determining fatigue is assessing whether an individual has kept their eyes closed for an extended duration within a specific timeframe. When provided with a video, our methodology initially utilizes the Multi-task Cascaded Convolutional Networks (MTCNN) to detect the position of faces in each frame. Subsequently, a convolutional neural network (CNN) is employed to determine whether the detected faces' eyes are open or closed. Finally, we conduct tests based on the dataset we have created. The results indicate that, despite the impact of face masks or eyeglass accessories on the model's performance in eye-open/eye-closed recognition, our model still achieves an accuracy of 92%. Regarding fatigue assessment, we utilize two fatigue conditions, 15% and 30% PERCLOS, for evaluation. In scenarios involving regular faces, the model achieves an accuracy of 98%. Furthermore, we conduct fatigue tests using an eye dataset. We observe that in videos with individuals wearing masks, occasional shifts in the eye feature positions detected by the face detection algorithm result in inaccurate eye extraction, thereby affecting the model's judgment.

論文審定書 ....................................................................................................................... i
誌謝 .................................................................................................................................. ii
摘要 ................................................................................................................................. iii
Abstract ............................................................................................................................ iv
第一章 緒論 .................................................................................................................... 1
第二章 文獻探討 ............................................................................................................ 3
第一節 疲勞偵測方式 ................................................................................................ 3
第二節 疲勞判斷標準PERCLOS ............................................................................ 10
第三節 眼部臉部資料集 .......................................................................................... 11
第三章 研究方法 .......................................................................................................... 13
第一節 資料集 .......................................................................................................... 13
第二節 眼睛閉闔狀態辨識 ...................................................................................... 19
第三節 疲勞認定 ...................................................................................................... 21
第四章 結果分析 .......................................................................................................... 24
第一節 Dlib和MTCNN比較 .................................................................................. 24
第二節 卷積神經網路開閉眼識別比較 .................................................................. 27
第三節 特殊情境比較 .............................................................................................. 29
第四節 疲勞判斷實驗 .............................................................................................. 31
第五章 結論與未來展望 .............................................................................................. 36
第一節 研究結論 ...................................................................................................... 36
第二節 未來展望 ...................................................................................................... 37
參考文獻 ........................................................................................................................ 39

[1] 吳宗修、陳冠旭 (2012), 疲勞駕駛事故與公路幾何設計關聯性研究, 101年道路交通安全與執法研討會
[2] LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation Applied to Handwritten ZIP Code Recognition. Neural Computation, 1 (4), pp. 541-551. doi: 10.1162/neco.1989.1.4.541.
[3] Reddy, B., Kim, Y. H., Yun, S., Seo, C., & Jang, J. (2017). Real-time Driver Drowsiness Detection for Embedded System Using Model Compression of Deep Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (pp. 121-128).
[4] Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters, 23 (10), pp. 1499-1503.
[5] Fagerberg, K. (2004). Vehicle-Based Detection of Inattentive Driving for Integration in an Adaptive Lane Departure Warning System Drowsiness Detection, (M.S. thesis). KTH Signals Sensors and Systems, Stockholm, Sweden.
[6] Mu, Z., Hu, J., & Min, J. (2017). Driver Fatigue Detection System Using Electroencephalography Signals Based on Combined Entropy Features. Applied Sciences, 7 (2), 150. https://doi.org/10.3390/app7020150.
[7] Massoz, Q., Langohr, T., François, C., & Verly, J. G. (2016). The ULg Multimodality Drowsiness Database (Called DROZY) and Examples of Use. In Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1-7). doi: 10.1109/WACV.2016.7477715.
[8] Soukupova, T., & Cech, J. (2016). Eye Blink Detection Using Facial Landmarks. In Proceedings of the 21st Computer Vision Winter Workshop (CVWW), Rimske Toplice, Slovenia.
[9] Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., & Hariri, B. (2020). YawDD: Yawning Detection Dataset. IEEE Dataport. https://dx.doi.org/10.21227/e1qm-hb90.
[10] United States. Federal Motor Carrier Safety Administration. Technology Division. (1998). PERCLOS: A Valid Psychophysiological Measure of Alertness as Assessed by Psychomotor Vigilance. https://doi.org/10.21949/1502740
[11] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems, 25, pp. 1097-1105.
[12] Weng, C. H., Lai, Y. H., & Lai, S. H. (2016). Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network. In Proceedings of the Asian Conference on Computer Vision (ACCV) Workshop on Driver Drowsiness Detection from Video, Taipei, Taiwan.
[13] Jabbar, R., Al-Khalifa, K., Kharbeche, M., Alhajyaseen, W., Jafari, M., & Jiang, S. (2018). Real-time Driver Drowsiness Detection for Android Application Using Deep Neural Networks Techniques. Procedia Computer Science, 130, pp. 400-407. ISSN 1877-0509.
[14] Wang, H. G., Zhu, Y., Chen, X. D., He, L. F., & Zheng, B. B. (2018). Hierarchical CNN-based Real-time Fatigue Detection System by Visual-based Technologies Using MSP Model. IET Image Processing, 12 (12), pp. 2319-2329.
[15] Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A Unified Embedding for Face Recognition and Clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 815-823).
[16] Zhao, Z., Zhou, N., Zhan, L., Yan, H., Xu, Y., & Zhang, Z. (2020). Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN. Computational Intelligence and Neuroscience, 2020, Article ID 7251280, 11. https://doi.org/10.1155/2020/7251280.
[17] 葉旻彥、黃揚倫、劉星瑜、傅翊慈、洪智力 (2009), 全時多感知主動式駕駛防禦裝置, 108年道路交通安全與執法研討會
[18] LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989). Backpropagation Applied to Handwritten ZIP Code Recognition. Neural Computation, 1 (4), pp. 541-551. doi: 10.1162/neco.1989.1.4.541.
[19] Song, F., Tan, X., Liu, X., & Chen, S. (2014). Eyes Closeness Detection from Still Images with Multi-scale Histograms of Principal Oriented Gradients. Pattern Recognition, 47 (9), pp. 2825-2838. https://doi.org/10.1016/j.patcog.2014.03.024.
[20] Zhuang, Q., Kehua, Z., Wang, J., & Chen, Q. (2020). Driver Fatigue Detection Method Based on Eye States With Pupil and Iris Segmentation. IEEE Access, 8, pp. 173440-173449. https://doi.org/10.1109/ACCESS.2020.3025818.
[21] Kazemi, V., & Sullivan, J. (2014). One Millisecond Face Alignment with an Ensemble of Regression Trees. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 1867-1874). Columbus, OH, USA: IEEE. doi: 10.1109/CVPR.2014.241.
[22] Simonyan, K., & Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition (arXiv:1409.1556).
[23] 郭永彩、李文濤、高潮 (2009) , 基於PERCLOS的駕駛員疲勞檢測算法, 計算計系統應用, 2009, 18 (8) , pp. 54-57
[24] 廖育昇、陳加增、陳育菘、高翔鈞 (2010),自適性人臉偵測於駕駛狀態監控系統之應用, 車輛研究測試中心, 經濟部技術處科專計畫 (99-EC-17-A-04-02-0803)
[25] 張袁 (2017), 基於眼電圖法的疲勞駕駛偵測, (碩士論文, 國立中央大學),臺灣博碩士論文知識加值系統, https://hdl.handle.net/11296/uvkjj9