Decoding video data. Receive a first video stream including a first plurality of frames encoded at a first resolution, a second video stream including a second plurality of frames encoded at a second resolution and a residual bitstream encoded at a third resolution. The second plurality of frames correspond to the first plurality of frames encoded at the first resolution and the first resolution is greater than the second resolution and the third resolution is at least between the first and second resolutions. Side information determined based at least on the first plurality of frames and the second plurality of frames. The video data decoded based at least on one received frame of the first plurality of frames, at least one received frame of the second plurality of frames encoded at the second resolution, the residual bitstream encoded at the third resolution, and the side information.