Near-fault ground motions can impose particularly high seismic demands on structures due to the pulses that are typically observed in the velocity time-histories. In this study it is empirically found that the critical response can be estimated from the directions corresponding to the maximum (max) or minimum (min) pulse-energy. Determination of the pulse-energy requires removing of the high-frequency content. For achieving this, the wavelet analysis and the least-square-fitting (LSF) algorithm are adopted. Results obtained by the two strategies are compared and differences between them are analyzed. Finally, the relationship between the critical response and the response derived from directions having the max or min pulse-energy confirms that using the pulse-energy for deriving the critical response of the building structures is reasonable.