Development of 3D head pose and gaze direction tracking method for driver assistance

Hitoshi Kubota, Hideo Saito

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Driver state monitoring is one of the key technologies for implementation of driver assistance systems. Drivers depend mainly on visual information as the basis of operating their vehicle. Therefore, detecting the driver's head pose and gaze direction would be an effective way to estimate the presence/absence of driving errors. This study focuses on using 3D movements of the head pose as the driver's state and proposes a method where the head pose is estimated from stereo images produced by multiple cameras installed around the dashboard in front of the driver's seat. We focus on the area around nostrils and estimate its 3D position from face images. Because, the camera installation location is restricted to the area around the dashboard and thus the face is photographed from the lower left/right angle, making the nostrils the most stable feature in this situation. First we search the nostril position using template matching and reconstruct its 3D position. Then we preciously estimate the 3D head pose by particle filtering with 3D model-based method. Fixation of the nostril 3D position is added to the motion model used for predicting the particle hypothesis. By reducing the size of the state space in this way and thus shortening the sampling interval, we are able to track the head pose with a high level of accuracy and at high speed even when the number of particles is small. In addition, we estimate the gaze direction by the vector between measured pupil center and estimated eyeball center. The 3D position of the pupil center is measured by stereo measurement using 2D coordinates extracted from each image. The 3D position of eyeball center is estimated with transformation from the initial position of eyeball model by the result of head pose estimation.

Original languageEnglish
Title of host publicationFISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety
Pages172-179
Number of pages8
Volume2
Publication statusPublished - 2008
Event32nd FISITA World Automotive Congress 2008 - Munich, Germany
Duration: 2008 Sep 142008 Sep 19

Other

Other32nd FISITA World Automotive Congress 2008
CountryGermany
CityMunich
Period08/9/1408/9/19

Fingerprint

Cameras
Template matching
Seats
Sampling
Monitoring

Keywords

  • Gaze direction tracking
  • Head pose tracking
  • Nostril
  • Particle filtering
  • Template matching

ASJC Scopus subject areas

  • Automotive Engineering
  • Safety, Risk, Reliability and Quality

Cite this

Kubota, H., & Saito, H. (2008). Development of 3D head pose and gaze direction tracking method for driver assistance. In FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety (Vol. 2, pp. 172-179)

Development of 3D head pose and gaze direction tracking method for driver assistance. / Kubota, Hitoshi; Saito, Hideo.

FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety. Vol. 2 2008. p. 172-179.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kubota, H & Saito, H 2008, Development of 3D head pose and gaze direction tracking method for driver assistance. in FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety. vol. 2, pp. 172-179, 32nd FISITA World Automotive Congress 2008, Munich, Germany, 08/9/14.
Kubota H, Saito H. Development of 3D head pose and gaze direction tracking method for driver assistance. In FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety. Vol. 2. 2008. p. 172-179
Kubota, Hitoshi ; Saito, Hideo. / Development of 3D head pose and gaze direction tracking method for driver assistance. FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety. Vol. 2 2008. pp. 172-179
@inproceedings{478cf5ff0b244c27b7d78f53f0c3e15b,
title = "Development of 3D head pose and gaze direction tracking method for driver assistance",
abstract = "Driver state monitoring is one of the key technologies for implementation of driver assistance systems. Drivers depend mainly on visual information as the basis of operating their vehicle. Therefore, detecting the driver's head pose and gaze direction would be an effective way to estimate the presence/absence of driving errors. This study focuses on using 3D movements of the head pose as the driver's state and proposes a method where the head pose is estimated from stereo images produced by multiple cameras installed around the dashboard in front of the driver's seat. We focus on the area around nostrils and estimate its 3D position from face images. Because, the camera installation location is restricted to the area around the dashboard and thus the face is photographed from the lower left/right angle, making the nostrils the most stable feature in this situation. First we search the nostril position using template matching and reconstruct its 3D position. Then we preciously estimate the 3D head pose by particle filtering with 3D model-based method. Fixation of the nostril 3D position is added to the motion model used for predicting the particle hypothesis. By reducing the size of the state space in this way and thus shortening the sampling interval, we are able to track the head pose with a high level of accuracy and at high speed even when the number of particles is small. In addition, we estimate the gaze direction by the vector between measured pupil center and estimated eyeball center. The 3D position of the pupil center is measured by stereo measurement using 2D coordinates extracted from each image. The 3D position of eyeball center is estimated with transformation from the initial position of eyeball model by the result of head pose estimation.",
keywords = "Gaze direction tracking, Head pose tracking, Nostril, Particle filtering, Template matching",
author = "Hitoshi Kubota and Hideo Saito",
year = "2008",
language = "English",
volume = "2",
pages = "172--179",
booktitle = "FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety",

}

TY - GEN

T1 - Development of 3D head pose and gaze direction tracking method for driver assistance

AU - Kubota, Hitoshi

AU - Saito, Hideo

PY - 2008

Y1 - 2008

N2 - Driver state monitoring is one of the key technologies for implementation of driver assistance systems. Drivers depend mainly on visual information as the basis of operating their vehicle. Therefore, detecting the driver's head pose and gaze direction would be an effective way to estimate the presence/absence of driving errors. This study focuses on using 3D movements of the head pose as the driver's state and proposes a method where the head pose is estimated from stereo images produced by multiple cameras installed around the dashboard in front of the driver's seat. We focus on the area around nostrils and estimate its 3D position from face images. Because, the camera installation location is restricted to the area around the dashboard and thus the face is photographed from the lower left/right angle, making the nostrils the most stable feature in this situation. First we search the nostril position using template matching and reconstruct its 3D position. Then we preciously estimate the 3D head pose by particle filtering with 3D model-based method. Fixation of the nostril 3D position is added to the motion model used for predicting the particle hypothesis. By reducing the size of the state space in this way and thus shortening the sampling interval, we are able to track the head pose with a high level of accuracy and at high speed even when the number of particles is small. In addition, we estimate the gaze direction by the vector between measured pupil center and estimated eyeball center. The 3D position of the pupil center is measured by stereo measurement using 2D coordinates extracted from each image. The 3D position of eyeball center is estimated with transformation from the initial position of eyeball model by the result of head pose estimation.

AB - Driver state monitoring is one of the key technologies for implementation of driver assistance systems. Drivers depend mainly on visual information as the basis of operating their vehicle. Therefore, detecting the driver's head pose and gaze direction would be an effective way to estimate the presence/absence of driving errors. This study focuses on using 3D movements of the head pose as the driver's state and proposes a method where the head pose is estimated from stereo images produced by multiple cameras installed around the dashboard in front of the driver's seat. We focus on the area around nostrils and estimate its 3D position from face images. Because, the camera installation location is restricted to the area around the dashboard and thus the face is photographed from the lower left/right angle, making the nostrils the most stable feature in this situation. First we search the nostril position using template matching and reconstruct its 3D position. Then we preciously estimate the 3D head pose by particle filtering with 3D model-based method. Fixation of the nostril 3D position is added to the motion model used for predicting the particle hypothesis. By reducing the size of the state space in this way and thus shortening the sampling interval, we are able to track the head pose with a high level of accuracy and at high speed even when the number of particles is small. In addition, we estimate the gaze direction by the vector between measured pupil center and estimated eyeball center. The 3D position of the pupil center is measured by stereo measurement using 2D coordinates extracted from each image. The 3D position of eyeball center is estimated with transformation from the initial position of eyeball model by the result of head pose estimation.

KW - Gaze direction tracking

KW - Head pose tracking

KW - Nostril

KW - Particle filtering

KW - Template matching

UR - http://www.scopus.com/inward/record.url?scp=84866140590&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84866140590&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84866140590

VL - 2

SP - 172

EP - 179

BT - FISITA World Automotive Congress 2008, Congress Proceedings - Vehicle Safety

ER -