To remove only noise components without removing desired signal components, a signal processing apparatus includes a noise decorrelator that removes noise signals having correlation between at least two input signals, in each of which a desired signal and a noise signal coexist, by receiving the at least two input signals from at least two channels, and a residual noise remover that removes residual noise included in an output signal of the noise decorrelator based on a phase difference between the output signal of the noise decorrelator and at least one input signal included in the at least two input signals.