An image processing apparatus includes a two-dimensional orthogonal transform unit configured to perform two-dimensional orthogonal transform on a plurality of images, an one-dimensional orthogonal transform unit configured to perform one-dimensional orthogonal transform in a direction in which the images are arranged on two-dimensional orthogonal transform coefficient data obtained by performing the two-dimensional orthogonal transform on the images using the two-dimensional orthogonal transform unit, and a three-dimensional orthogonal transform coefficient data encoder configured to encode three-dimensional orthogonal transform coefficient data obtained by performing the one-dimensional orthogonal transform on the two-dimensional orthogonal transform coefficient data using the one-dimensional orthogonal transform unit.