2022年南港瓶蓋工廠所舉行的Maker Faire 中，在公司的攤位上，最受到矚目與歡迎的，就是Google MediaPipe 的深蹲偵測互動遊戲了，達到一定的深蹲次數不僅買徽章可以折價，還可以免費獲得摩艾石像3D列印一個，吸引了很多大小朋友，一起來看看當時熱鬧的畫面吧!   [embedyt] https://www.youtube.com/watch?v=sMeqfwYyZoE[/embedyt]    

原理及功能說明

本次的深蹲互動專題，以這篇[累累累] Google Mediapipe 深蹲偵測，結合 Arduino 首次接觸就上手所改編的，一樣著重於偵測身體大腿及小腿的角度小於一定的數值，將深蹲次數顯示出來外，並取消了和Arduino的連線。而MediaPipe已經將人體簡化成由點和線所構成的人體線條骨架的樣貌，如下圖所示。   由上圖所示，就是偵測左腿點23(left_hip)到點25(left _knee)的直線距離，和點25(left _knee)到點27(left _ankle)的直線距離，兩條直線所產生的夾角小於120；右腿則是點24(right_hip)到點26(right _knee)的直線距離，和點28(right _knee)到點27(right _ankle)的直線距離，兩條直線所產生的夾角小於120。 也就是說，當左右腳膝蓋彎曲角度小於120度時，字體顏色就會變綠色。只要螢幕上顯示的字體「Left Angle」、「Right Angle」、「Pose」皆為綠色時，次數會加1。   再來是遊戲互動體驗，筆者使用了一顆超大的Enter鍵，當程式開始執行時，畫面會出現指示「Press Enter To Start」，Enter鍵按下去後就可以開始深蹲遊戲，此時就會開始計算深蹲次數。再按一次Enter鍵，次數會歸零，同時又會回到「Press Enter To Start」的畫面。而筆者在每次計算深蹲次數後，都會加上音效，以確保動作完成，如以下所示。   再來是場地架設，以當時在Maker Faire南港瓶蓋工廠場地為例，筆者使用的是羅技C270的攝影鏡頭，鏡頭距離地面高度需2公尺，距離人的身體約3公尺，才能偵測得到全身骨架畫面，由下圖所示。  

程式撰寫

由於程式過長，故說明重要片段，如下列程式所示。 以下程式為影像訊號來源。第1行為攝影機訊號；第2行可撥放影片。 [pastacode lang="python" manual="cam%20%3D%20cv2.VideoCapture(3)%0A%23cam%3Dcv2.VideoCapture(%22POSE1.mp4%22)" message="" highlight="" provider="manual"/]   以下程式為計算身體比率函式。 [pastacode lang="python" manual="def%20get_body_ratio(landmarks)%3A%0A%C2%A0%C2%A0%C2%A0%20r_body%20%3D%20abs(landmarks%5Bmppose.PoseLandmark%5B%22RIGHT_SHOULDER%22%5D.value%5D.y%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20-%20landmarks%5Bmppose.PoseLandmark%5B%22RIGHT_HIP%22%5D.value%5D.y)%0A%09%0A%C2%A0%C2%A0%C2%A0%20l_body%20%3D%20abs(landmarks%5Bmppose.PoseLandmark%5B%22LEFT_SHOULDER%22%5D.value%5D.y%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0-%20landmarks%5Bmppose.PoseLandmark%5B%22LEFT_HIP%22%5D.value%5D.y)%0A%09%0A%C2%A0%C2%A0%C2%A0%20avg_body%20%3D%20(r_body%20%2B%20l_body)%20%2F%202%0A%0A%C2%A0%C2%A0%C2%A0%20r_leg%20%3D%20abs(landmarks%5Bmppose.PoseLandmark%5B%22RIGHT_HIP%22%5D.value%5D.y%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20-%20landmarks%5Bmppose.PoseLandmark%5B%22RIGHT_ANKLE%22%5D.value%5D.y)%0A%0A%C2%A0%C2%A0%C2%A0%20l_leg%20%3D%20abs(landmarks%5Bmppose.PoseLandmark%5B%22LEFT_HIP%22%5D.value%5D.y%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20-%20landmarks%5Bmppose.PoseLandmark%5B%22LEFT_ANKLE%22%5D.value%5D.y)%0A%0A%C2%A0%C2%A0%C2%A0%20if%20r_leg%20%3E%20l_leg%3A%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20return%20r_leg%20%2F%20avg_body%0A%C2%A0%C2%A0%C2%A0%20else%3A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20return%20l_leg%20%2F%20avg_body" message="" highlight="" provider="manual"/]   以下程式為計算左右腳的角度函式。 [pastacode lang="python" manual="def%20get_knee_angle(landmarks)%3A%0A%0A%C2%A0%C2%A0%C2%A0%20r_hip%20%3D%20get_landmark(landmarks%2C%20%22RIGHT_HIP%22)%0A%C2%A0%C2%A0%C2%A0%20l_hip%20%3D%20get_landmark(landmarks%2C%20%22LEFT_HIP%22)%0A%0A%C2%A0%C2%A0%C2%A0%20r_knee%20%3D%20get_landmark(landmarks%2C%20%22RIGHT_KNEE%22)%0A%C2%A0%C2%A0%C2%A0%20l_knee%20%3D%20get_landmark(landmarks%2C%20%22LEFT_KNEE%22)%0A%0A%C2%A0%C2%A0%C2%A0%20r_ankle%20%3D%20get_landmark(landmarks%2C%20%22RIGHT_ANKLE%22)%0A%C2%A0%C2%A0%C2%A0%20l_ankle%20%3D%20get_landmark(landmarks%2C%20%22LEFT_ANKLE%22)%0A%0A%C2%A0%C2%A0%C2%A0%20r_angle%20%3D%20calc_angles(r_hip%2C%20r_knee%2C%20r_ankle)%0A%C2%A0%C2%A0%C2%A0%20l_angle%20%3D%20calc_angles(l_hip%2C%20l_knee%2C%20l_ankle)%0A%0A%0A%C2%A0%C2%A0%C2%A0%20%23print(r_hip)%0A%C2%A0%C2%A0%C2%A0%20m_hip%20%3D%20(r_hip%20%2B%20l_hip)%0A%C2%A0%C2%A0%C2%A0%20m_hip%20%3D%20%5Bx%20%2F%202%20for%20x%20in%20m_hip%5D%0A%C2%A0%C2%A0%C2%A0%20m_knee%20%3D%20(r_knee%20%2B%20l_knee)%0A%C2%A0%C2%A0%C2%A0%20m_knee%20%3D%20%5Bx%20%2F%202%20for%20x%20in%20m_knee%5D%0A%C2%A0%C2%A0%C2%A0%20m_ankle%20%3D%20(r_ankle%20%2B%20l_ankle)%0A%C2%A0%C2%A0%C2%A0%20m_ankle%20%3D%20%5Bx%20%2F%202%20for%20x%20in%20m_ankle%5D%0A%0A%C2%A0%C2%A0%C2%A0%20mid_angle%20%3D%20calc_angles(m_hip%2C%20m_knee%2C%20m_ankle)%0A%0A%C2%A0%C2%A0%C2%A0%20return%20%5Bint(r_angle)%2Cint(l_angle)%20%2Cint(mid_angle)%5D" message="" highlight="" provider="manual"/]   以下程式為計算按下Enter鍵的次數，可觸發遊戲開始。 [pastacode lang="python" manual="%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20if%20(read_dir_key%20%3D%3D%2013)%3A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20sport%5B'count'%5D%20%3D%200%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20run_flag%20%2B%3D%201%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20if%20(run_flag%20%3D%3D%202)%3A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20run_flag%20%3D%200" message="" highlight="" provider="manual"/]   以下程式為左腳膝蓋彎曲角度之顏色顯示。小於120度綠色，介於120~130度黃色，大於130度紅色。 而檔案POSE_entercount_sound.py中的第226~248行則為顯示右腳膝蓋彎曲角度之顏色顯示。 [pastacode lang="python" manual="if%20knee_angles%5B0%5D%20%3C%20120%3A%0A%0A%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(82%2C%20169%2C%20255)%2C%204%2C%20cv2.LINE_AA)%0A%0A%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(0%2C%20255%2C%200)%2C%201%2C%20cv2.LINE_AA)%0A%0A%0A%09elif%20knee_angles%5B0%5D%20%3C%20130%3A%0A%0A%09%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(82%2C%20169%2C%20255)%2C%204%2C%20cv2.LINE_AA)%0A%0A%0A%09%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(0%2C%20255%2C%20255)%2C%201%2C%20cv2.LINE_AA)%0A%0A%09%09else%3A%0A%0A%09%09%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(82%2C%20169%2C%20255)%2C%204%2C%20cv2.LINE_AA)%0A%0A%09%09%09cv2.putText(preview%2C%20%22Left%20Angle%3A%20%7B%3Ad%7D%22.format(knee_angles%5B0%5D)%2C%20(400%2C%20360)%0A%09%09%09%09%09%09%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%200.8%2C%20(0%2C%200%2C%20255)%2C%201%2C%20cv2.LINE_AA)%0A%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20" message="" highlight="" provider="manual"/]   決定深蹲的次數，如果次數已加1，則會撥放音效。 [pastacode lang="python" manual="if%20status%3A%0A%09if%20avg_angle%20%3E%20160%3A%0A%09%09status%20%3D%20False%0A%09%09pass_time%20%3D%20time.time()%20-%20start_time%0A%09%09start_time%20%3D%200%0A%0A%09%09if%203000%20%3E%20pass_time%20%3E%200.5%3A%0A%09%09%09sport%5B'count'%5D%20%3D%20sport%5B'count'%5D%20%2B%201%0A%0A%09%09%09while%20(pygame.mixer.music.get_busy()!%3D1)%3A%0A%09%09%09%09pygame.mixer.music.load('coin05.mp3')%0A%09%09%09%09pygame.mixer.music.play()%0A%0A%09%09%09%09sport%5B'calories'%5D%20%3D%20sport%5B'calories'%5D%20%2B%20int(0.66%20*%20pass_time)%0A%09%09%09%09logger(sport%5B'count'%5D%2C%20sport%5B'calories'%5D)%0A%09%09%09%09tmp%20%3D%20f%22a%7Bsport%5B'count'%5D%7D%5Cn%22%0A%09%09%09%09%23ser.write(str.encode(tmp))%0A%09%09%09%09tmp%20%3D%20f%22b%7Bsport%5B'calories'%5D%7D%5Cn%22%0A%09%09%09%09%23ser.write(str.encode(tmp))%0A%09%09%09%09%23print(pass_time)%0A%0A%0A%09%09%09%09else%3A%0A%09%09%09%09%09if%20avg_angle%20%3C%20120%20and%20body_ratio%20%3C%201.2%3A%0A%09%09%09%09%09%09start_time%20%3D%20time.time()%0A%09%09%09%09%09%09status%20%3D%20True" message="" highlight="" provider="manual"/]     在螢幕上顯示"Press Enter To Start"，以開始深蹲遊戲 [pastacode lang="python" manual="cv2.putText(preview%2C%20%22Press%20Enter%20To%20Start%22%2C%20(10%2C%2080)%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%201.5%2C%20(0%2C%200%2C%200)%2C%205%2C%20cv2.LINE_AA)%0A%0Acv2.putText(preview%2C%20%22Press%20Enter%20To%20Start%22%2C%20(10%2C%2080)%0A%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%C2%A0%20%2C%20cv2.FONT_HERSHEY_SIMPLEX%2C%201.5%2C%20(0%2C%20255%2C%200)%2C%202%2C%20cv2.LINE_AA)" message="" highlight="" provider="manual"/]   以上程式傳送門-連結 下載解壓縮後，電腦中需依本文安裝Python的執行環境(操作至圖 21)。

執行程式

[pastacode lang="python" manual="cd%20pose_detection%0Apython%20POSE_entercount_sound.py%0A" message="" highlight="" provider="manual"/]  

影片展示

以下影片為腦波弱老闆在南港瓶蓋工廠所示範的深蹲畫面，一起來看看吧! [embedyt] https://www.youtube.com/watch?v=VS4oZzdNI-I[/embedyt]