An IR-UWB system can achieve high resolution in ranging because it uses a very short pulse with a duration of less than 1 [ns]. In order to reduce cost and power consumption, a ranging system with high-speed comparators has been proposed. In this system, it is necessary to reduce noise power through averaging the comparator outputs. In this paper, the effect of clock offset on the averaging process is evaluated, and a two-stage averaging scheme for the IR-UWB ranging system is proposed. Through computer simulation, it is proved that the proposed scheme can reduce the effect of the clock offset with appropriate numbers of initial averaging.