A reconstructed image which further reflects the photo shooting information of a main object is generated. A layer determiner defines a layer of a reconstructed image. A layer image generator reconstructs an image of an object included in the allocated depth range from a light field image and the depth map of the light field image for each layer. A conversion pixel extractor extracts corresponding pixels on a conversion layer which corresponds to an object to be modified. The object to be modified is designated by an operation acquired by a modification operation acquirer. A reconstructed image generator converts layer images using a conversion matrix defined by a conversion matrix determiner and generates an image whose composition has been modified by superimposing the layer images.