A coupled model based on crystal plasticity and phase-field theories that express both plastic anisotropy of HCP metals and expansion/shrinkage of twin-bands is proposed in the present study. In this model, the difference of the hardening rate in each slip system is expressed by changing their dislocation mobility as a numerical parameter defined in the crystal plasticity framework. The stress calculated via crystal plasticity analysis becomes to the driving force of multi-phase-filed equations that express the evolution of twin bands of several variants, which include both the growth and shrinkage. Solving this equation set, the rate of twinning/detwinning and the mirror-transformed crystal basis in the twinned/detwinned phase are obtained and then, crystal plasticity analysis is carried out again. Using the present model, a uniaxial cyclic loading simulation along  direction on the specimen including two variants of twin-bands is carried out by means of finite element method (FEM). The results show that the detwinning stress decreases with increase of the pre-tensioned strain. This is caused by a residual compression stress resulting from the twin shearing that occurs in the vicinity of two twin boundaries approaching each other.