本論文介紹了一種三維手勢追蹤技術，使用智慧型裝置的二維相機並結合Wi-Fi訊號注入鎖定都卜勒雷達系統，這項技術的動作偵測原理涉及像素基底的計算機視覺演算法以及都卜勒相位移偵測技術。本論文提出了三維手勢軌跡之空間座標校正程序，將相機中像素基底的平面影像座標轉換為真實空間中的座標，藉此得到手勢橫向位移軌跡資訊，再使用Wi-Fi訊號都卜勒相移偵測雷達系統，從Wi-Fi的反射訊號所含手勢造成的都卜勒相位移，得到手勢縱向位移軌跡資訊，最終結合手勢軌跡的橫向和縱向分量，即可以建構出空間座標中的三維手勢軌跡。
在計算機視覺處理方面，本研究使用的是動態追蹤的技術，藉由辨識動作與背景模型的差異，而得到手掌的重心位置，但陸續遇到一些問題需要進一步修改演算法，最終選用的是基於HSL色彩之區塊辨別演算法，而在雷達技術方面，則使用了單基式被動雷達系統，首先完成了雙手與一維手勢辨識的應用，其後搭配軟體技術與計算機視覺處理完成了三維手勢軌跡之辨識，並分別發展出非即時性與即時性的手勢追蹤技術，達成混合手勢感測雷達之應用。
在手勢辨識的技術中，混合手勢感測雷達技術比當前主流技術更具有發展潛力，低計算資源與低功耗是此雷達架構之優勢，並且微型化後能應用在智慧型手機之中，不需要額外的相機鏡頭組或是高頻訊號源，就能達成AR或是VR的體感互動功能。

This thesis presents a 3-D hand gesture tracking technique using the 2D camera of a smartphone and injection-locked Doppler radar system based on Wi-Fi signals. The motion detection principle of this technique involves methods of pixel-based computer vision and the Doppler phase-shift detection. This thesis proposed a spatial coordinate calibration algorithm to obtain the information of the transverse displacement component of the hand gesture by converting the pixel coordinates of the image in the smartphone into the space coordinates of the radar. On the other hand, the Doppler phase shift caused by hand gestures was extracted from the Wi-Fi reflection signal by using the Wi-Fi signal-based radar system to obtain the information of the longitudinal displacement component of the same hand gesture. Finally, the 3-D hand gesture trajectory was constructed in space coordinates by combining the transverse and longitudinal displacement components.
In terms of computer vision processing, the dynamic tracking technique was used in early stage, the center of the palm was identified by the difference between the moving target and the background model. But there were some issues that needed to modify the algorithm further. At last, the blocks identification algorithm based on the HSL-color was used in follow-up experiments. In terms of the radar technique, the monostatic passive radar system was used to perform 1-D and both hands gesture detections first, and then the 3-D gesture trajectory model was accomplished with the help of signal processing and computer vision. Eventually, developments of non-real-time and real-time gesture tracking techniques were achieved with the hybrid system.
Talking about the technology of 3-D hand gesture recognition (HGR), hybrid gesture sensing technique has higher potential than the current mainstream optical methods. Advantages of this hybrid detection method are lower computation resources and power consumption than the other. After miniaturization, the proposed system can be used in the smartphone to achieve AR or VR functions and doesn’t need extra cameras or RF transmission source for 3-D gesture detection applications.

目錄
論文審定書...i
誌謝...ii
摘要...iii
ABSTRACT...iv
目錄...vi
圖次...viii
表次...xii
第一章 緒論...1
1.1 研究背景與動機...1
1.2 三維攝影技術與雷達技術之介紹...2
1.3 論文章節組織...7
第二章 混合手勢感測雷達之機制原理...8
2.1 計算機視覺處理機制...8
2.1.1 計算機視覺之演算法...8
2.1.2 影像座標與真實世界座標...14
2.2 WI-FI訊號都卜勒相移偵測雷達原理...20
2.2.1 應用於雷達系統之注入鎖定理論...20
2.2.2 都卜勒相移偵測雷達之手勢追蹤技術...25
第三章 WI-FI訊號都卜勒相移偵測雷達架構...31
3.1 雷達架構之優化...31
3.1.1 頻率選擇之探討...32
3.1.2 方向性天線探討...34
3.1.3 系統雜訊比之探討...40
3.1.4 致動器實驗對照組...44
3.2 都卜勒相移偵測技術應用...49
第四章 混合手勢感測雷達之應用...53
4.1 非即時性三維手勢軌跡追蹤技術...53
4.1.1 真實座標的軌跡校正...53
4.1.2 整合計算機視覺之三維手勢軌跡...59
4.2 即時性三維手勢軌跡追蹤技術...64
第五章 結論...71
參考文獻...72
 
圖次
圖1. 1...ASUS ZENFONE AR之攝像硬體規格...2
圖1. 2...KINECT FOR WINDOWS（A）KINECT V1感應器（B）KINECT V2感應器...3
圖1. 3...REALSENSE F200感測器...4
圖1. 4...LEAP MOTION 感測器...5
圖1. 5...混合手勢感測雷達示意圖...6
圖2. 1...起始確認連接端口之介面...8
圖2. 2...動態偵測演算法之介面...9
圖2. 3...卡爾曼濾波後軌跡與原始軌跡比較...10
圖2. 4...相機之有效偵測範圍距離之實驗 （A）手掌比例過小（B）初始手勢過於接近螢幕...11
圖2. 5...手勢移動後的軌跡偏移 （A）手勢初始移動之軌跡（B）手勢轉折後之...軌跡...11
圖2. 6...重心以手肘為主時判斷的手勢軌跡變化（A）正常的手勢軌跡情況（B）手肘對軌跡的影響情況...12
圖2. 7...HSL 色彩空間示意圖 （A）HSL色彩空間（B）HSL色彩空間截面...13
圖2. 8...初始化區塊偵測之介面...14
圖2. 9...HSL色彩空間的閥值調整介面...14
圖2. 10...二維影像之座標定義...15
圖2. 11...真實物體的位置投影到螢幕畫面的位置...16
圖2. 12...相機參數轉換機制...17
圖2. 13...影像平面中XY軸夾角相對的傾斜程度...17
圖2. 14...外部參數轉換示意圖...18
圖2. 15...影像座標系統...19
圖2. 16...真實世界座標系統...20
圖2. 17...注入鎖定振盪器之簡化模型...21
圖2. 18...注入訊號向量分析圖...21
圖2. 19...不同注入功率下的鎖入時間與頻率關係...25
圖2. 20...傳統連續波都卜勒雷達架構...26
圖2. 21...ILQR解調單元...27
圖2. 22...WI-FI訊號都卜勒相移偵測雷達示意圖...27
圖3. 1...64QAM調制訊號注入鎖定之頻域表現（A）ESG產生之調制訊號（B）注入鎖定後之輸出訊號...31
圖3. 2...注入鎖定振盪器之輸出頻率與WI-FI頻段...32
圖3. 3...振盪器鎖定在2.46 GHZ時的正交通道之訊息...33
圖3. 4...振盪器鎖定在特定頻率時的正交通道訊息 （A）操作頻率為2.40 GHZ...（b）操作頻率為2.43GHz ...34
圖3. 5...振盪器鎖定在特定頻率時的正交通道訊息 （A）操作頻率為2.37 GHZ...（b）操作頻率為2.55GHz ...34
圖3. 6...量測環境示意圖...35
圖3. 7...高指向性天線...35
圖3. 8...高指向性天線量測之不同手勢正交通道資訊...36
圖3. 9...高指向性天線量測之不同手勢封包振幅...36
圖3. 10...高指向性天線量測之不同手勢位移偵測...36
圖3. 11...低指向性天線...37
圖3. 12...低指向性天線量測之不同手勢正交通道資訊...37
圖3. 13...低指向性天線量測之不同手勢封包振幅...37
圖3. 14...低指向性天線量測之不同手勢位移偵測...38
圖3. 15...全向性天線...38
圖3. 16...全向性天線量測之不同手勢正交通道資訊...38
圖3. 17...全向性天線量測之不同手勢封包振幅...39
圖3. 18...全向性天線量測之不同手勢位移偵測...39
圖3. 19...移動平均之衰減與頻率關係...41
圖3. 20... ADS分析整體訊號的直流位準...41
圖3. 21... ADS分析手勢訊號之基底位準...42
圖3. 22...HP-8594E頻譜分析儀...43
圖3. 23...ZABER致動器A-LSQ150A...44
圖3. 24... ZABER CONSOLE之操作介面...45
圖3. 25...致動器測量環境...46
圖3. 26...致動器連續擺動3 CM的正交通道資訊...46
圖3. 27...致動器連續擺動3 CM的封包值與位移...47
圖3. 28...致動器週期擺動3 CM的正交通道資訊...47
圖3. 29...致動器週期擺動3 CM的封包值與位移...48
圖3. 30... A-LSQ150A致動器之誤差值...49
圖3. 31...WI-FI訊號都卜勒相移偵測雷達實體圖...49
圖3. 32... LABVIEW 程式流程圖...50
圖3. 33...手勢造成相位角逐漸遞減的現象...50
圖3. 34...達成手勢讓收機翻頁的實驗...51
圖3. 35...都卜勒相移雙手偵測雷達架構...51
圖3. 36...應用雷達技術玩橫向捲軸闖關之遊戲...52
圖4. 1...WI-FI訊號都卜勒相移偵測雷達架構...53
圖4. 2...應用計算機視覺技術來捕獲手勢的橫向軌跡...54
圖4. 3...手勢橫向位移資訊...54
圖4. 4...手勢縱向位移資訊（A）正交都卜勒訊號（B）瞬時位移資訊...55
圖4. 5...SENSOR KINETICS之軟體介面...56
圖4. 6...校正程序示意圖（A）朝著已知角ΘX的方向揮動（B）朝著已知角ΘY的方向揮動...57
圖4. 7...X軸軌跡校正實驗...58
圖4. 8...真實座標之橫向位移轉換關係 （A）X軸位移轉換參數（B）Y軸位移轉換參數...58
圖4. 9...混合手勢雷達實驗...59
圖4. 10...控制手機端與電腦端的傳輸時間差異...60
圖4. 11...LABVIEW處理三維手勢訊號之流程圖...60
圖4. 12...非即時性LABVIEW程式介面概覽...61
圖4. 13...橫向位移接收程序之人機介面...61
圖4. 14...使用MATLAB建立手勢三維軌跡...62
圖4. 15...混合手勢感測雷達技術重建三維帶狀手勢軌跡（A）連續拍攝手勢揮動（b）重建後的三維手勢軌跡...63
圖4. 16...即時性混合手勢感測雷達“SET AND REPORT”使用者介面...65
圖4. 17...即時性混合手勢感測雷達“Z RAW DATA”使用者介面...66
圖4. 18...即時性混合手勢感測雷達“3D REAL TIME”使用者介面...66
圖4. 19...即時性混合手勢感測雷達之程式介面一...67
圖4. 20...即時性混合手勢感測雷達之程式介面二...68
圖4. 21...即時性混合手勢感測雷達與LEAP MOTION的比對實驗...68
圖4. 22...LEAP MOTION數值擷取的程式介面...69
圖4. 23...即時性混合手勢感測雷達與LEAP MOTION之手勢軌跡對照...70

表次
表2. 1...ZE552KL之簡易規格表...8
表2. 2...向量分析圖之參數定義...22
表3. 1...距離天線10-30 CM處手勢資訊...42
表3. 2...距離天線20-30 CM處手勢資訊...42
表3. 3...訊雜比與注入訊號間的關係...43
表3. 4...注入訊號功率與平均訊雜比之關係...44
表4. 1...即時性混合手勢感測雷達與LEAP MOTION之三維手勢軌跡距離誤差...69

[1] R. Azuma, “A Survey of Augmented Reality,” Presence: Teleoperators and Virtual Environments, pp. 355–385, August 1997
[2] ZenFone AR應用Google AR、VR之相關技術 [Online]. Available: https://www.mobile01.com/ne-wsdetail/21636/zenfone-ar-google-daydream-tango-ar-vr-unbox
[3] Microsoft Corporation with Kinect for Windows [Online] Available: https://developer.microsoft.com/zh-tw/windows/kinect
[4] PrimeSense WIKI [Online] Available: https://en.wikipedia.org/wiki/PrimeSense
[5] L. Xia, C.-C. Chen, and J. Aggarwal, “Human detection using depth information by kinect,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference on june 2011, pp. 15 –22.
[6] Zhou Ren, Junsong Yuan, Jingjing Meng, Zhengyou Zhang, Robust Part-Based Hand Gesture Recognition Using Kinect Sensor, Multimedia, IEEE Transactions on 15.5 (2013), pp.lllO-1l20 2013.
[7] J. Han, L. Shao, D. Xu, and J. Shotton, “Enhanced computer vision with microsoft kinect sensor: A review,” IEEE Trans. Cybern., vol. 43, no. 5, pp. 1318–1334, Oct. 2013
[8] Intel Corporation with RealSense [Online] Available: https://www.intel.com.tw/content/www/tw/zh/architecture-and-technology/realsense-overview.html
[9] Liang-Gee Chen et al., " Interactive Sticker System with Intel RealSense," Consumer Electronics (ICCE), Las Vegas, NV, USA, 2017, pp. 174-175.
[10] V.Silva et al., "Real-time emotions recognition system," 2016 8th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), Lisbon, Portugal, 2016, pp. 201-206.
[11] Leap Motion, Inc with Leap Motion [Online] Available: https://www.leapmotion.com
[12] C.-H. Chuan, E. Regina, and C. Guardino, “American Sign Language Recognition Using Leap Motion Sensor,” 2014 13th International Conference on Machine Learning and Applications, pp. 541–544, 2014.
[13] G. Marin, F. Dominio, and P. Zanuttigh. “Hand gesture recognition with leap motion and Kinect devices,” In International Conference on Image Processing (ICIP), 2014.
[14] F. Adib and D. Katabi, “See through walls with Wi-Fi,” in Proc. ACM Special Interest Group on Data Commun. Conf., Hong Kong, China, Aug. 2013, pp. 75-86.
[15] Google’s Project Soli. [online] Available: https://atap.google.com/soli/(accessed 5 Dec. 2016).
[16] F. Adib, C.-Y. Hsu, H. Mao, D. Katabi, and F. Durand, “Capturing the human figure through a wall,” ACM Trans. Graph., vol. 34, no. 6, pp. 219:1- 13, Nov. 2015.
[17] Q. Pu, S. Gupta, S. Gollakota, and S. Patel, “Whole-home gesture recognition using wireless signals,” in Proc. Int. Conf. Mobile Computing Networking, 2013, pp. 27–38.
[18] C. Stauffer and W.E.L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Int. Conf. Computer Vision Pattern Recog., 1999, pp. 246-252.
[19] HSL and HSV WIKI [Online] Available: https://en.wikipedia.org/wiki/HSL_and_HSV
[20] Tefen Lin; Bo-Hao Liao; Su-Ling Hsu; Jie Wang, “Experimental Investigation of HSL Color Model in Error Diffusion,” 2015 8th International Conference on Ubi-Media Computing (UMEDIA), pp. 268-272
[21] 吳佩珊，應用圓形格點於相機校正之實驗研究，中華大學機械與航太工程研究所碩士論文，民國九十一年。
[22] 劉正豐，基於高解析度彩色和低解析深度攝影機之高品質的虛擬視角影像合成技術，國立中正大學工學院電機工程研究所碩士論文，民國一百年。
[23] Kheresy Space OpenNI的座標系統 [Online] Available: https://kheresy.wordpress.com/2012/04/05/coordinate-system-in-openni/
[24] R. Adler, “A study of locking phenomena in oscillators,” Proc. IRE, vol. 34, no. 6, pp. 351–357, Jun. 1946.
[25] T. H. Hee and A. Hajimiri, “Oscillator Phase Noise: A Tutorial,” IEEE J. Solid-State Circuits, vol. 35, no. 3, pp. 326-336.
[26] F.-K. Wang, C.-J. Li, C.-H. Hsiao, T.-S. Homg, J. Lin, K.-C. Peng, J.-K. Jau, J.-Y. Li, and c.-c. Chen, "A novel vital-sign sensor based on a self-injection-locked oscillator," IEEE Trans. Microw. Theory Tech., vol. 58, pp. 4112-4120, Dec. 2010.
[27] 方佳暉，含有自我注入鎖定振盪器之有源標籤用於室內定位生理訊號，國立中山大學電機工程學系碩士論文，民國一百零二年。
[28] G. D. Vendelin, A. M. Pavio, and U. L. Rohde, Microwave Circuit Design Using Linear and Nonlinear Techniques. New York: Wiley, 1990, Chapter 6.
[29] L. J. Paciorek, “Injection locking of oscillators,” Proc. IEEE, vol. 53, no. 11, pp. 1723–1727, Nov. 1965.
[30] R. C. Mackey, “Injection locking of klystron oscillators,” IRE Trans. Microwave Theory and Tech., vol. 10, no. 4, pp. 228–235, July 1962.
[31] C.-T. Chen, C.-H. Hsiao, T.-S. Horng, K.-C. Peng, and C.-J. Li, “Cognitive polar receiver using two injection-locked oscillator stages,” IEEE Trans. Microw. Theory Tech., vol. 59, no. 12, pp. 3484-3493, Dec. 2011.
[32] O. Boric-Lubecke, Y. Nikawa, W. Snyder, J. Lin, K. Mizuno "Skin properties at millimeter waves", Proc. Asia Pacific Microwave Conference 1998 APMC'98, Vol. 2, pp. 877- 880, Dec. 1998.
[33] A.-D. Droitcour, V.-M. Lubecke, J. Lin, and O.-B. Lubecke, “A Microwave radio for Doppler radar sensing of vital signs,” in IEEE MTT-S Int. Microwave Symp. Dig., Phoenix, AZ, May 2001, pp. 175-178.
[34] C. Li, J. Cummings, J. Lam, E. Graves and W. Wu, “Radar remote monitoring of vital signs,” IEEE Microwave Mag., vol. 10, no. 1, pp. 47-56, Feb. 2009.
[35] B. Razavi, “Design considerations for direct-conversion receivers,” IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 44, no. 6, pp. 428-435, Jun. 1997.
[36] C.-T. Chen, C.-H. Hsiao, T.-S. Horng, and K.-C. Peng, “Wireless polar receiver using two injection-locked oscillator stages,” in IEEE MTT-S Int. Microw. Symp. Dig., Baltimore, MD, Jun. 2011, pp. 1–4, Session WE1D–3.
[37] B. Razavi, “A study of injection locking and pulling in oscillators,” IEEE J. Solid-State Circuits, vol. 39, no. 9, pp. 1415–1424, Sep. 2004.
[38] M. E. Heidari and A. A. Abidi, “Behavioral models of frequency pulling in oscillators,” in 2007 IEEE Int. Behavioral Modeling and Simulation Workshop, pp.100–104.
[39] A. D. Droitcour, O. Boric-Lubecke, V. M. Lubecke, J. Lin, and G. T. A. Kovac, “Range correlation and I/Q performance benefits in single-chip silicon Doppler radars for noncontact cardiopulmonary monitoring,” IEEE Trans. Microw. Theory Tech, vol. 52, no. 3, pp. 838-848, Mar. 2004.
[40] H.-R. Chuang, Y.-F. Chen, and K.-M. Chen, “Automatic clutter-canceler for microwave life-detection systems,” IEEE Trans. Instrum. Mea., vol. 40, no. 4, pp. 747-750, Aug. 1991.
[41] T.-Y. Chin, K.-Y. Lin, S.-F. Chang and C.-C. Chang, “A fast clutter cancellation method in quadrature Doppler radar for noncontact vital signal detection,” in IEEE MTT-S Int. Microw. Symp. Dig., Anaheim, CA, May 2010, pp. 764-767.
[42] A. Singh and V.-M. Lubecke, “A heterodyne receiver for harmonic Doppler radar cardio-pulmonary monitoring with body-worn passive RF tags,” in IEEE MTT-S Int. Microwave Symp. Dig., Anaheim, CA, Jun. 2010, pp. 1600-1603.
[43] B.-K. Park, D. Samardzija, V.-M. Lubecke, O.-B. Lubecke and T. Sizer, “DC offset reduction in phase diversity heterodyne Doppler radar system,” in IEEE AP-S Int. Symp. Dig., Honolulu, HI, Jun. 2007, pp. 3884-3887.
[44] B. Park, O. Boric-Lubecke, and V.M. Lubecke, “Arctangent demodulation with DC offset compensation in quadrature Doppler radar receiver systems,” IEEE Trans. Microwave Theory Tech., vol. 55, no. 5, pp. 1073-1079, May 2007.
[45] M.C. Budge, Jr. and M.P. Burt, “Range Correlation Effects on Phase and Amplitude Noise,” Proceedings of IEEE Southeastcon, 1993.
[46] M. C. Budge and M. P. Burt, “Range correlation effects in radars,” in Proc. Rec. IEEE Nat. Radar Conf., Apr. 1993, pp. 212–216.
[47] Fuqin Xiong and Monty Andro. “The Effect of Doppler Frequency Shift, Frequency Offset of the Local Oscillators, and Phase Noise on the Performance of Coherent OFDM Receivers,” NASA document
[48] 邱彥禎，利用行動通訊之都卜勒效應感測手勢與生命徵象，國立中山大學電機工程學系碩士論文，民國一百零三年。
[49] Antenna-Theory Specific Absorption Rate (SAR) [Online] Available: http://www.antenna-theory.com/definitions/sar.php
[50] S. W. Smith, “Moving Average Filters,” in The Scientist and Engineer's Guide to Digital Signal Processing, 1st ed. California: California Technical Pub, 1997, pp.277-284
[51] ]S.-J. Kweon, S.-H. Shin, S.-H. Jo, and H.-J. Yoo, “Reconfigurable high-order moving-average filter using inverter-based variable transconductance amplifiers,” IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 61, no. 12, pp. 942–946, Dec. 2014
[52] F.K. Wang, T.-S. Horng, K.-C. Peng, J.-K. Jau, J.-Y. Li and C.-C. Chen, “Mutual injection-locked SIL sensor array for vital sign detection with random body movement cancellation,” in IEEE MTT-S Int. Microw. Symp. Dig., Jun. 2011, pp. TH2C-4: 1-4.
[53] Zaber Technologies Inc. Zaber WIKI [Online] Available：https://www.zaber.com/products/product_detail.php?detail=A-LSQ150A
[54] Zaber Technologies Inc. Zaber WIKI [Online] Available：http://www.zaber.com/wiki/So-ftware/Zaber_Console
[55] David M. Pozar, Microwave Engineering, 4th Edition
[56] R. Vaughan, “Polarization diversity in mobile communications,” IEEE Trans. Veh. Technol., vol. 39, no. 3, pp. 177–186, Mar. 1990.'
[57] National Instruments Inc. DAQ NI-9215 [Online] Available：http://sine.ni.com/nips/cds/view/p/lang/zht/nid/208793
[58] A. Fetic, D. Juric, D. Osmankovic, “The procedure of a camera calibration using Camera Calibration Toolbox for MATLAB MIPRO,” 2012 Proceedings of the 35th International Convention. 2012:1752-1757.
[59] F.-K. Wang, M.-C. Tang, Y.-C. Chiu, and T.-S. Horng, “Gesture sensing using retransmitted wireless communication signals based on Doppler radar technology,” IEEE Trans. Microw. Theory Techn., vol. 63, no. 12, pp. 4592–4602, Dec. 2015.
[60] Y. Kim and H. Ling, ‘‘Human activity classification based on microDoppler signatures using a support vector machine,’’ IEEE Trans. Geosci. Remote Sens., vol. 47, no. 5, pp. 1328–1337, May 2009.
[61] T. Kopinski, S. Geisler, and U. Handmann, “Gesture-based human-machine interaction for assistance systems,” 2015 IEEE International Conference on., pp. 510-517, Aug. 2015