Development of health monitoring system of construction becomes an important task for structure damage identification. Since the building structure has many indeterminable factors, the method using modal analysis to identify the structure damage is of less precision. A method applying the grey system to the structural damage identification is presented. The grey system of the frequency variance rate and the stiffness change is established with the first-order single argument grey system (GM(1,1)). GM(1,1) prediction is used to reflect the global function of the structural dynamic fingerprints and find out the relation of the frequency variance rate and the stiffness change. Vibration tests for frame structures were carried out, with considered many cases, including single-damage and multi-damage with different degrees and locations. The results show that for shear buildings, damage degree and location can be determined by using the grey system created through measuring the frequency change.