A shake detection technique which does not mistakenly detect a swing-back component at the end of panning as a shake is provided. A shake detection apparatus includes: a sensor which detects velocity or angular velocity; a high-pass filter which removes a low-frequency component from an output signal of the sensor; a shake detection device which detects shake of an apparatus body based on an output signal from the high-pass filter; a determination device which determines whether the output signal of the high-pass filter contains a swing-back component outputted when a shake continued in one direction stops; and a removal device which removes the swing-back component from the output signal of the high-pass filter based on a determination result of the determination device.