The optimal buffering strategy of the present invention allows for full smooth reverse functionality in an MPEG-type stream player, while reducing the buffering requirements. Furthermore, the buffering strategy drastically reduces the required number of passes through the video data unit by optimal scheduling of picture decodes. A video data unit buffering strategy for smooth reverse includes the following operations. First, a video data unit is scanned before decoding any pictures within the video data unit. Positions of each picture are then marked out in the video data unit. Next, the entire video data unit is decoded, except the open GOP pictures which need a reference picture from an adjacent video data unit. The open GOP pictures are then copied to the end of the current video data unit in the buffer, and the new video data unit continues to be fetched.