skeh/head-pose-estimation
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
No description
98 commits 4 branches 0 tags 139 MiB
PureBasic 99.9%
Find a file
Yin Guobing 1e66826c22
Update README.md
2018-09-26 16:20:33 +08:00
assets add 3D model of 68 points 2018-01-25 19:02:15 +08:00
.gitignore Initial commit 2017-12-27 15:23:36 +08:00
demo.gif update demo gif 2018-01-26 08:47:32 +08:00
demo1.gif add more demo gif 2018-01-26 08:57:01 +08:00
estimate_head_pose.py add OS checking 2018-02-22 16:03:38 +08:00
LICENSE Initial commit 2017-12-27 15:23:36 +08:00
mark_detector.py more concise expression of image shape 2018-01-30 16:42:40 +08:00
optical_flow_tracker.py use optical flow tracker to update kalman filter parameters 2017-12-29 10:26:37 +08:00
os_detector.py typo fix: -> multiprocessing 2018-02-22 16:04:01 +08:00
pose_estimator.py update annotation drawing function to accept custom line width 2018-01-25 14:22:07 +08:00
README.md Update README.md 2018-09-26 16:20:33 +08:00
stabilizer.py bug fix: initialize parameters according to state_num 2018-01-12 13:37:17 +08:00

README.md

Head pose estimation

This repo shows how to estimate human head pose from videos using TensorFlow and OpenCV.

demo demo

Dependence

  • TensorFlow 1.4
  • OpenCV 3.3
  • Python 3

The code is tested under Ubuntu 16.04.

How it works

There are three major steps:

  1. Face detection. A face detector is adopted to provide a face box containing a human face. Then the face box is expanded and transformed to a square to suit the needs of later steps.

  2. Facial landmark detection. A custom trained facial landmark detector based on TensorFlow is responsible for output 68 facial landmarks.

  3. Pose estimation. Once we got the 68 facial landmarks, a mutual PnP algorithms is adopted to calculate the pose.

Miscellaneous

  • The marks is detected frame by frame, which result in small variance between adjacent frames. This makes the pose unstaible. A Kalman filter is used to solve this problem, you can draw the original pose to observe the difference.

  • The 3D model of face comes from OpenFace, you can find the original file here.

  • The build in face detector comes from OpenCV. https://github.com/opencv/opencv/tree/master/samples/dnn/face_detector

License

The code is licensed under the MIT license. However, the pre-trained TensorFlow model file is trained with various public data sets which have their own licenses. Please refer to them before using this code.

(Currently we are trying to empower embeded devices to be more powerful with deep learning, please get in touch if you are interested and would like to join us.)