A method of processing a signal is disclosed. The present invention includes receiving a downmix signal generated from plural channel signal and spatial information indicating attribute of the plural channel signal to upmix the downmix signal; obtaining inter-channel phase difference (IPD) coding flag indicating whether IPD value is used to the spatial information from header of the spatial information; obtaining IPD mode flag based on the IPD coding flag from the frame of the spatial information, the IPD mode flag indicating whether the IPD value is used to a frame of the spatial information; obtaining the IPD value of parameter band of parameter time slot in the frame, based on the IPD mode flag; smoothing the IPD value by modifying the IPD value by using IPD value of previous parameter time slot; and generating plural channel signal by applying the smoothed IPD value to the downmix signal.