An image processing apparatus includes an input unit receiving an encoded image data stream; a decoding processor decoding the stream to obtain image data; a first storage storing the image data; a second storage storing the image data and having a storage area storing at least one picture whose size is the largest of image data to be decoded; a controller analyzing the stream and storing the image data in the first and second storages according to an analysis result; and an output unit reading and outputting the stored pictures in a display order. The controller sets storage areas in the first storage according to image size information, stores the image data in the storage areas, determines whether the image size changes according to the analysis result, and, when the image size changes, stores image data of a picture displayed immediately before the image size changes in the second storage.