Lip reading system using novel Japanese visemes classification and hierarchical weighted discrimination

Shinsuke Okita, Yasue Mitsukura, Nozomu Hamada

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In recent years, automatic lip reading based on 'visemes' have been studied by researchers for realizing human-machine interactive communication system in many applications. However there are a lot of problems such as the definition of the number of viseme classes, discrimination method of visemes, speech recognition method based on visemes, and so on. In this paper, a novel classification of Japanese visemes and hierarchical weighted discrimination method for speech recognition are proposed to address these problems. We augmented the classification number of visemes from 6(conventional) to 9 to represent the words in more detailed by visemes. In addition, considering the difficulty in discriminating with increase of the number of visemes, the hierarchical weighted discrimination method is proposed. For the purpose of comparing with the conventional method, the ATR phonetically balanced word group, which is large vocabulary and includes various visemes, was used and applied to word recognition experiments. From these results, we confirmed the proposed method worked well.

Original languageEnglish
Title of host publicationISPACS 2013 - 2013 International Symposium on Intelligent Signal Processing and Communication Systems
Pages531-536
Number of pages6
DOIs
Publication statusPublished - 2013 Dec 1
Event2013 21st International Symposium on Intelligent Signal Processing and Communication Systems, ISPACS 2013 - Naha, Okinawa, Japan
Duration: 2013 Nov 122013 Nov 15

Publication series

NameISPACS 2013 - 2013 International Symposium on Intelligent Signal Processing and Communication Systems

Other

Other2013 21st International Symposium on Intelligent Signal Processing and Communication Systems, ISPACS 2013
CountryJapan
CityNaha, Okinawa
Period13/11/1213/11/15

    Fingerprint

Keywords

  • Image processing
  • Pattern recognition
  • lip reading
  • visemes mouth-shape code
  • visual speech recognition

ASJC Scopus subject areas

  • Artificial Intelligence
  • Signal Processing

Cite this

Okita, S., Mitsukura, Y., & Hamada, N. (2013). Lip reading system using novel Japanese visemes classification and hierarchical weighted discrimination. In ISPACS 2013 - 2013 International Symposium on Intelligent Signal Processing and Communication Systems (pp. 531-536). [6704608] (ISPACS 2013 - 2013 International Symposium on Intelligent Signal Processing and Communication Systems). https://doi.org/10.1109/ISPACS.2013.6704608