Provided is a technology which enables early removal from a tracking target of an object unnecessary to be tracked and which, together therewith, when the tracking target is temporarily in a state of being not detected, enables continuous tracking of the relevant tracking target without removing it from the tracking target.A detection unit 2 detects, as a tracking candidate, an object included in an image by using image processing. A corresponding degree calculation unit 3 calculates a corresponding degree which is index representing a probability that the detected tracking candidate corresponds to the tracking target registered as an object to be tracked. When having determined that it is unnecessary to track the tracking target based on an evaluation value calculated by using the corresponding degree, a decision unit 4 deletes the registration of the object to be tracked with respect to the relevant tracking target.