A server computing device hosts one or more virtual machines. A first virtual machine corresponding to a first client device receives a first media stream that includes first content corresponding to a plurality of frames of video data and generates a first digest segment that corresponds to the first media stream. The first digest segment includes a representation of the plurality of frames but does not include the video data. The first virtual machine stores the first digest segment in a cache at the server system. A second virtual machine corresponding to a second client device receives a playback position of the first media stream playing at the second client device and uses the playback position from the second client device and the first digest segment stored in the cache to perform processing to recreate a representation of the playback of the first media stream on the second client device.