skeh
/
head-pose-estimation
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
132 Commits 4 Branches 0 Tags 139 MiB
PureBasic 99.9%
Go to file
yinguobing 981cac645e update mark model with new weights 2021-03-09 18:00:15 +08:00
assets update mark model with new weights 2021-03-09 18:00:15 +08:00
doc add wechat info 2019-09-23 11:31:24 +08:00
.gitignore Initial commit 2017-12-27 15:23:36 +08:00
LICENSE Initial commit 2017-12-27 15:23:36 +08:00
README.md update reademe 2020-12-17 17:39:53 +08:00
estimate_head_pose.py hide the face box 2020-12-17 18:50:30 +08:00
mark_detector.py update model file 2020-12-17 16:50:28 +08:00
optical_flow_tracker.py use optical flow tracker to update kalman filter parameters 2017-12-29 10:26:37 +08:00
os_detector.py fixed typo 2020-06-16 09:49:13 +08:00
pose_estimator.py add function to draw axes 2019-08-14 09:36:38 +08:00
stabilizer.py fixed typo 2020-06-16 09:44:53 +08:00

README.md

Head pose estimation

Real time human head pose estimation using TensorFlow and OpenCV.

demo demo

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

TensorFlow OpenCV Numpy

The code was tested on Ubuntu 20.04.

Installing

This repository already provided a pre-trained model for facial landmarks detection. Just git clone then you are good to go.

# From your favorite development directory:
git clone https://github.com/yinguobing/head-pose-estimation.git

Running

A video file or a webcam index should be assigned through arguments. If no source provided, the built in webcam will be used by default.

With video file

For any video format that OpenCV supports (mp4, avi etc.):

python3 estimate_head_pose.py --video /path/to/video.mp4

With webcam

The webcam index should be provided:

python3 estimate_head_pose.py --cam 0

How it works

There are three major steps:

  1. Face detection. A face detector is introduced to provide a face bounding box containing a human face. Then the face box is expanded and transformed to a square to suit the needs of later steps.

  2. Facial landmark detection. A pre-trained deep learning model take the face image as input and output 68 facial landmarks.

  3. Pose estimation. After getting 68 facial landmarks, the pose could be calculated by a mutual PnP algorithm.

The marks are detected frame by frame that makes the pose unstable. A Kalman filter is used to solve this problem, you can draw the original pose to observe the difference.

Retrain the model

To retrain the facial landmark detection model please refer to this series of posts (in Chinese only). The training code is also open sourced: https://github.com/yinguobing/cnn-facial-landmark

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Authors

Yin Guobing (尹国冰) - yinguobing

Acknowledgments

The pre-trained TensorFlow model file is trained with various public data sets which have their own licenses. Please refer to them before using this code.

The 3D model of face comes from OpenFace, you can find the original file here.

The build in face detector comes from OpenCV. https://github.com/opencv/opencv/tree/master/samples/dnn/face_detector