An image processing apparatus includes an image acquisition unit configured to extract an image from a video image, a history management unit configured to generate movement history of an object, a setting unit configured to set a search range in the image based on the movement history, and a tracking unit configured to detect a target object in the search range and associate a currently detected target object with a previously detected target object, to perform tracking on the target object.