Most currently available damage detection methods are global in nature, i.e., the dynamic properties (natural frequencies and mode shapes) are obtained for the entire structure from the input-output data using a global structural analysis. However, natuml frequencies and mode shapes are not sensitive to minor damage and local damage. The techniques using time-domain dynamic responses are appealing and promising. Furthermore, the dynamic responses of structures under environmental excitation or small- scale earthquakes are very economical information for structural identification and health monitoring, especially in the place where small-scale earthquakes occur very frequently. Some information about structural parameters and dynamic properties can be identified by the direct use of these time-domain response. There is an approach by directly using dynamic responses in time series without extraction of dynamic properties, which used acceleration, velocity and displacement time histories as the input of the emulator neural network, called acceleration-based emulator neural network (AENN) for free vibration. In this paper, the AENN is extended beyond the limitation of free vibration to forced vibration. The acceleration time histories, which are readily available in real structures, are only required. This is the first phase, through which the damage occurrence alarm can be obtained by observing the relative root mean square (RRMS) error between the output of AENN and the real value. After knowing the damage occurrence, the next phase is to localize and quantify the damage. Most currently available damage localization approaches are using pattern recognition methods to classify the different damage location. However, such approaches need analytical data for all damage case situations, which can be computationally expensive and even impossible. Therefore, the system identification is utilized. Seeing as the currently available system identification techniques are normally based on the frequency-domain approach, so in this paper the system identification problem is transferred to optimization problem with the convenience for time-domain. The particle swarm optimization (PSO) is utilized. The proposed approach is carried out in two phases as briefly described in Fig. 1. We analyze two five-story shear the structure to verify the proposed method experimentally.