In this paper, a lips extraction method that can extract lips region from varying lips shape at the moment of speech by using only one template image is described. The method that is proposed in this paper, has invariance for an open and closed mouth, showing or not showing any teeth, and has high speed and high extraction accuracy in consideration for characteristics of the lips by using a genetic method. This method uses the template matching using a genetic algorithm. Furthermore, colour of lips and characteristics of the lips shape variances at the moment of speech in this system are utilized. The effectiveness of this method is demonstrated with only one template for each person being tested and a search object, that is, the varying lips shape at the moment of speech of vowels by means of computer simulations. These computer simulations indicate that this method can extract the varying lips shape at the moment of speech by using only one template. Moreover, in the extraction processing of every vowel, a high speed and high extraction accuracy can be obtained.