Temporal and fine-grained pedestrian action recognition on driving recorder database

Hirokatsu Kataoka, Yutaka Satoh, Yoshimitsu Aoki, Shoko Oikawa, Yasuhiro Matsui

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01% on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23% on the near-miss driving recorder database (NDRDB). The paper has improved +8.28% and +6.53% from baseline two-stream fusion convnets.

Original languageEnglish
Article number627
JournalSensors (Switzerland)
Volume18
Issue number2
DOIs
Publication statusPublished - 2018 Feb 20

Fingerprint

recorders
Databases
vehicles
Advanced driver assistance systems
walking
traffic
proposals
emerging
safety
Fusion reactions
fusion
Pedestrians
Recognition (Psychology)
Walking
estimates
configurations
Safety

Keywords

  • Advanced driver-assistance systems (ADAS)
  • Driving recorder
  • Fine-grained pedestrian action recognition
  • Two-stream convnets

ASJC Scopus subject areas

  • Analytical Chemistry
  • Atomic and Molecular Physics, and Optics
  • Biochemistry
  • Instrumentation
  • Electrical and Electronic Engineering

Cite this

Temporal and fine-grained pedestrian action recognition on driving recorder database. / Kataoka, Hirokatsu; Satoh, Yutaka; Aoki, Yoshimitsu; Oikawa, Shoko; Matsui, Yasuhiro.

In: Sensors (Switzerland), Vol. 18, No. 2, 627, 20.02.2018.

Research output: Contribution to journalArticle

Kataoka, Hirokatsu ; Satoh, Yutaka ; Aoki, Yoshimitsu ; Oikawa, Shoko ; Matsui, Yasuhiro. / Temporal and fine-grained pedestrian action recognition on driving recorder database. In: Sensors (Switzerland). 2018 ; Vol. 18, No. 2.
@article{d76386ae0c0e4a3f8a639681ed56cb3b,
title = "Temporal and fine-grained pedestrian action recognition on driving recorder database",
abstract = "The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01{\%} on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23{\%} on the near-miss driving recorder database (NDRDB). The paper has improved +8.28{\%} and +6.53{\%} from baseline two-stream fusion convnets.",
keywords = "Advanced driver-assistance systems (ADAS), Driving recorder, Fine-grained pedestrian action recognition, Two-stream convnets",
author = "Hirokatsu Kataoka and Yutaka Satoh and Yoshimitsu Aoki and Shoko Oikawa and Yasuhiro Matsui",
year = "2018",
month = "2",
day = "20",
doi = "10.3390/s18020627",
language = "English",
volume = "18",
journal = "Sensors (Switzerland)",
issn = "1424-8220",
publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",
number = "2",

}

TY - JOUR

T1 - Temporal and fine-grained pedestrian action recognition on driving recorder database

AU - Kataoka, Hirokatsu

AU - Satoh, Yutaka

AU - Aoki, Yoshimitsu

AU - Oikawa, Shoko

AU - Matsui, Yasuhiro

PY - 2018/2/20

Y1 - 2018/2/20

N2 - The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01% on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23% on the near-miss driving recorder database (NDRDB). The paper has improved +8.28% and +6.53% from baseline two-stream fusion convnets.

AB - The paper presents an emerging issue of fine-grained pedestrian action recognition that induces an advanced pre-crush safety to estimate a pedestrian intention in advance. The fine-grained pedestrian actions include visually slight differences (e.g., walking straight and crossing), which are difficult to distinguish from each other. It is believed that the fine-grained action recognition induces a pedestrian intention estimation for a helpful advanced driver-assistance systems (ADAS). The following difficulties have been studied to achieve a fine-grained and accurate pedestrian action recognition: (i) In order to analyze the fine-grained motion of a pedestrian appearance in the vehicle-mounted drive recorder, a method to describe subtle change of motion characteristics occurring in a short time is necessary; (ii) even when the background moves greatly due to the driving of the vehicle, it is necessary to detect changes in subtle motion of the pedestrian; (iii) the collection of large-scale fine-grained actions is very difficult, and therefore a relatively small database should be focused. We find out how to learn an effective recognition model with only a small-scale database. Here, we have thoroughly evaluated several types of configurations to explore an effective approach in fine-grained pedestrian action recognition without a large-scale database. Moreover, two different datasets have been collected in order to raise the issue. Finally, our proposal attained 91.01% on National Traffic Science and Environment Laboratory database (NTSEL) and 53.23% on the near-miss driving recorder database (NDRDB). The paper has improved +8.28% and +6.53% from baseline two-stream fusion convnets.

KW - Advanced driver-assistance systems (ADAS)

KW - Driving recorder

KW - Fine-grained pedestrian action recognition

KW - Two-stream convnets

UR - http://www.scopus.com/inward/record.url?scp=85042333992&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042333992&partnerID=8YFLogxK

U2 - 10.3390/s18020627

DO - 10.3390/s18020627

M3 - Article

VL - 18

JO - Sensors (Switzerland)

JF - Sensors (Switzerland)

SN - 1424-8220

IS - 2

M1 - 627

ER -