Hierarchical human action recognition around sleeping using obscured posture information

Yuta Kudo, Takehiko Sashida, Yoshimitsu Aoki

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

This paper presents a new approach for human action recognition around sleeping with the human body parts locations and the positional relationship between human and sleeping environment. Body parts are estimated from the depth image obtained by a time-of-flight (TOF) sensor using oriented 3D normal vector. Issues in action recognition of sleeping situation are the demand of availability in darkness, and hiding of the human body by duvets. Therefore, the extraction of image features is difficult since color and edge features are obscured by covers. Thus, first in our method, positions of four parts of the body (head, torso, thigh, and lower leg) are estimated by using the shape model of bodily surface constructed by oriented 3D normal vector. This shape model can represent the surface shape of rough body, and is effective in robust posture estimation of the body hidden with duvets. Then, action descriptor is extracted from the position of each body part. The descriptor includes temporal variation of each part of the body and spatial vector of position of the parts and the bed. Furthermore, this paper proposes hierarchical action classes and classifiers to improve the indistinct action classification. Classifiers are composed of two layers, and recognize human action by using the action descriptor. First layer focuses on spatial descriptor and classifies action roughly. Second layer focuses on temporal descriptor and classifies action finely. This approach achieves a robust recognition of obscured human by using the posture information and the hierarchical action recognition.

Original languageEnglish
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
PublisherSPIE
Volume9534
ISBN (Print)9781628416992
DOIs
Publication statusPublished - 2015
Event12th International Conference on Quality Control by Artificial Vision - Le Creusot, France
Duration: 2015 Jun 32015 Jun 5

Other

Other12th International Conference on Quality Control by Artificial Vision
CountryFrance
CityLe Creusot
Period15/6/315/6/5

Fingerprint

posture
Action Recognition
Descriptors
Classifiers
human body
classifiers
Normal vector
Availability
thigh
torso
Color
darkness
Classify
Classifier
Sensors
Time-of-flight
Robust Estimation
availability
Human
beds

Keywords

  • Hierarchical action classes and classifiers
  • Human action recognition
  • Obscured 3D posture information
  • Sleeping situation

ASJC Scopus subject areas

  • Applied Mathematics
  • Computer Science Applications
  • Electrical and Electronic Engineering
  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics

Cite this

Kudo, Y., Sashida, T., & Aoki, Y. (2015). Hierarchical human action recognition around sleeping using obscured posture information. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 9534). [953418] SPIE. https://doi.org/10.1117/12.2182870

Hierarchical human action recognition around sleeping using obscured posture information. / Kudo, Yuta; Sashida, Takehiko; Aoki, Yoshimitsu.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 9534 SPIE, 2015. 953418.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Kudo, Y, Sashida, T & Aoki, Y 2015, Hierarchical human action recognition around sleeping using obscured posture information. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 9534, 953418, SPIE, 12th International Conference on Quality Control by Artificial Vision, Le Creusot, France, 15/6/3. https://doi.org/10.1117/12.2182870
Kudo Y, Sashida T, Aoki Y. Hierarchical human action recognition around sleeping using obscured posture information. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 9534. SPIE. 2015. 953418 https://doi.org/10.1117/12.2182870
Kudo, Yuta ; Sashida, Takehiko ; Aoki, Yoshimitsu. / Hierarchical human action recognition around sleeping using obscured posture information. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 9534 SPIE, 2015.
@inproceedings{f009a8866f5842b396e77bfacdce10aa,
title = "Hierarchical human action recognition around sleeping using obscured posture information",
abstract = "This paper presents a new approach for human action recognition around sleeping with the human body parts locations and the positional relationship between human and sleeping environment. Body parts are estimated from the depth image obtained by a time-of-flight (TOF) sensor using oriented 3D normal vector. Issues in action recognition of sleeping situation are the demand of availability in darkness, and hiding of the human body by duvets. Therefore, the extraction of image features is difficult since color and edge features are obscured by covers. Thus, first in our method, positions of four parts of the body (head, torso, thigh, and lower leg) are estimated by using the shape model of bodily surface constructed by oriented 3D normal vector. This shape model can represent the surface shape of rough body, and is effective in robust posture estimation of the body hidden with duvets. Then, action descriptor is extracted from the position of each body part. The descriptor includes temporal variation of each part of the body and spatial vector of position of the parts and the bed. Furthermore, this paper proposes hierarchical action classes and classifiers to improve the indistinct action classification. Classifiers are composed of two layers, and recognize human action by using the action descriptor. First layer focuses on spatial descriptor and classifies action roughly. Second layer focuses on temporal descriptor and classifies action finely. This approach achieves a robust recognition of obscured human by using the posture information and the hierarchical action recognition.",
keywords = "Hierarchical action classes and classifiers, Human action recognition, Obscured 3D posture information, Sleeping situation",
author = "Yuta Kudo and Takehiko Sashida and Yoshimitsu Aoki",
year = "2015",
doi = "10.1117/12.2182870",
language = "English",
isbn = "9781628416992",
volume = "9534",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",

}

TY - GEN

T1 - Hierarchical human action recognition around sleeping using obscured posture information

AU - Kudo, Yuta

AU - Sashida, Takehiko

AU - Aoki, Yoshimitsu

PY - 2015

Y1 - 2015

N2 - This paper presents a new approach for human action recognition around sleeping with the human body parts locations and the positional relationship between human and sleeping environment. Body parts are estimated from the depth image obtained by a time-of-flight (TOF) sensor using oriented 3D normal vector. Issues in action recognition of sleeping situation are the demand of availability in darkness, and hiding of the human body by duvets. Therefore, the extraction of image features is difficult since color and edge features are obscured by covers. Thus, first in our method, positions of four parts of the body (head, torso, thigh, and lower leg) are estimated by using the shape model of bodily surface constructed by oriented 3D normal vector. This shape model can represent the surface shape of rough body, and is effective in robust posture estimation of the body hidden with duvets. Then, action descriptor is extracted from the position of each body part. The descriptor includes temporal variation of each part of the body and spatial vector of position of the parts and the bed. Furthermore, this paper proposes hierarchical action classes and classifiers to improve the indistinct action classification. Classifiers are composed of two layers, and recognize human action by using the action descriptor. First layer focuses on spatial descriptor and classifies action roughly. Second layer focuses on temporal descriptor and classifies action finely. This approach achieves a robust recognition of obscured human by using the posture information and the hierarchical action recognition.

AB - This paper presents a new approach for human action recognition around sleeping with the human body parts locations and the positional relationship between human and sleeping environment. Body parts are estimated from the depth image obtained by a time-of-flight (TOF) sensor using oriented 3D normal vector. Issues in action recognition of sleeping situation are the demand of availability in darkness, and hiding of the human body by duvets. Therefore, the extraction of image features is difficult since color and edge features are obscured by covers. Thus, first in our method, positions of four parts of the body (head, torso, thigh, and lower leg) are estimated by using the shape model of bodily surface constructed by oriented 3D normal vector. This shape model can represent the surface shape of rough body, and is effective in robust posture estimation of the body hidden with duvets. Then, action descriptor is extracted from the position of each body part. The descriptor includes temporal variation of each part of the body and spatial vector of position of the parts and the bed. Furthermore, this paper proposes hierarchical action classes and classifiers to improve the indistinct action classification. Classifiers are composed of two layers, and recognize human action by using the action descriptor. First layer focuses on spatial descriptor and classifies action roughly. Second layer focuses on temporal descriptor and classifies action finely. This approach achieves a robust recognition of obscured human by using the posture information and the hierarchical action recognition.

KW - Hierarchical action classes and classifiers

KW - Human action recognition

KW - Obscured 3D posture information

KW - Sleeping situation

UR - http://www.scopus.com/inward/record.url?scp=84931308192&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84931308192&partnerID=8YFLogxK

U2 - 10.1117/12.2182870

DO - 10.1117/12.2182870

M3 - Conference contribution

SN - 9781628416992

VL - 9534

BT - Proceedings of SPIE - The International Society for Optical Engineering

PB - SPIE

ER -