An image processing device including a subject frame setting section which, by operating a subject detector which detects a subject captured in an image, sets a subject frame which surrounds a predetermined range of the subject detected from the image; an acceptance frame setting section which sets an acceptance frame with a range wider than the subject frame according to the context of the image; a position detecting section which detects a specified position on an image which is specified by a user; and a recognizing section which recognizes a subject which is a tracking target based on the acceptance frame set by the acceptance frame setting section and the specified position detected by the position detecting section.