An image processing apparatus for performing registration processing on a plurality of images by calculating a motion vector between the plurality of images includes a motion vector measurement region setting unit that sets a plurality of motion vector measurement regions for measuring the motion vector, a motion vector calculation unit that calculates motion vectors of the plurality of motion vector measurement regions, a motion vector reliability calculation unit that calculates a reliability of the motion vectors of the plurality of motion vector measurement regions on the basis of information relating to a local magnification of an optical system used to obtain the images, and a motion vector integration processing unit that determines an inter-image motion vector by integrating the motion vectors of the plurality of motion vector measurement regions in accordance with the reliability of the motion vectors.