When searching for a registered image similar to a query image, an image processing apparatus estimates an area having a difference by comparing images determined to be similar if a plurality of similar images is found at the time of search of a similar image using an image feature amount in order to enhance search accuracy. Then, the image processing apparatus compares the query image with the plurality of images similar to the query image based on the difference area.