An encoding unit encodes moving image data so that it has a temporal hierarchical structure in which a frame rate in a case of decoding up to a layer that is one above a predetermined layer is higher than a frame rate in a case of decoding up to the predetermined layer. A first acquisition unit acquires a frame rate of encoded first moving image data. A second acquisition unit acquires second moving image data targeted for encoding and a recording frame rate that is designated by a user operation. The encoding unit encodes the second moving image data so that it has a temporal hierarchical structure including a layer corresponding to the frame rate of the first moving image data and a layer corresponding to the recording frame rate.