According to one embodiment an image encoder encodes original image data using reference image data stored in a frame memory. The image encoder includes a controller, a memory, a motion controller, and a motion compensation module. The controller generates control information including a starting point coordinate and an ending point coordinate of a necessary area in the reference image data stored in the frame memory. The memory includes a storage area in which at least part of the reference image data is stored. The motion controller determines whether the necessary area is a transfer-necessary area or a transfer-unnecessary area based on the control information, and transfers the reference image data of the transfer-necessary area from the frame memory to the storage area. The motion compensation module generates prediction image data using the reference image data stored in the storage area.