An image synthesizing device for synthesizing a second image of an object in a predetermined illumination condition from a first image of the object includes an illumination condition estimation section for estimating an illumination condition from a luminance value of the first image, and a normal information estimation section estimates normal information containing a normal direction of the object, with regard to a predetermined pixel in the first image. An error component information estimation section estimates information of an error component in the predetermined illumination condition, an image synthesizing section synthesizes the second image of the object in the predetermined illumination condition, and the normal information estimation section estimates normal information of a predetermined pixel from information of a plurality of the pixels in the first image.