Improving diagnosis performance by matching the appearances of images while maintaining difference arising from the difference in tissue structure without being influenced by a non-interest region. Two medical images are divided into a plurality of tissue regions, each with each tissue imaged therein, and a similarity level is calculated for each corresponding pair of the plurality of tissue regions between the two medical images based on the form of each tissue region. An interest level is set for each corresponding pair and a weigh is obtained for each corresponding pair according to the similarity level and interest level. An image processing condition is set such that the matching level of an image characteristic of each corresponding pair is varied according to the weight, and image processing is performed on at least either one of the two medical images according to the set image processing condition.