Using the virtual data-driven measurement to support the prototyping of hand gesture recognition interface with distance sensor

Chengshuo Xia, Ayane Saito, Yuta Sugiura

Research output: Contribution to journalArticlepeer-review

Abstract

The performance of a fixed-sensor-based hand gesture recognition system is typically influenced by the position and number of sensors. The traditional development approach to hand gesture recognition systems follows a process of sensor pre-deployment, data collection, and model training, which is highly time-consuming and expensive to calculate how many sensors to place where, and the system has low secondary development flexibility. In this paper, we present a new development flow to assist in prototyping distance sensor-based gesture recognition interfaces. The designed system was able to simulate the position and number of sensors to recognize gestures. Using a reconstructed hand motion, the virtual distance sensor generated simulated signals and trained a convolutional neural network model. In a real-world setting, the sensor system only needed to be configured by transfer learning to recognize gestures at the same sensor layout. The proposed method was able to indicate sensor configuration and the trained classifier via virtual distance data, which can effectively reduce the development cost. We evaluated two prototype interfaces of the proposed method using distance sensors and demonstrated that the system effectively provided deployment recommendations, and models trained using virtual measurement data could effectively recognize real gestures.

Original languageEnglish
Article number113463
JournalSensors and Actuators A: Physical
Volume338
DOIs
Publication statusPublished - 2022 May 1

Keywords

  • Hand gesture recognition
  • Human-computer interface
  • Infrared distance sensor
  • Interface prototyping
  • Virtual sensor

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Instrumentation
  • Condensed Matter Physics
  • Surfaces, Coatings and Films
  • Metals and Alloys
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Using the virtual data-driven measurement to support the prototyping of hand gesture recognition interface with distance sensor'. Together they form a unique fingerprint.

Cite this