In this paper, dynamic characteristics of a medium-height office building with passive dampers is investigated. The building considered here is equipped with two kinds of passive dampers, viscous vibration control walls and low yield point steel-based buckling resistant braces. Eleven accelerometers are installed on basement, 1st, 5th, 10th and 14th floor. Two displacement meters for measuring the response of dampers are installed. To know the real performance of the building, we consider not only horizontal components but also vertical components. Dynamic characteristics of the building under several earthquakes are analyzed using multi-input and multi-output models for system identification. Using acceleration responses of multiple observation points, dynamic characteristics are estimated more precisely. The estimation errors of modal parameters are obtained simultaneously. Applying the method, it is indicated that vertical input has a significant influence on translational components.