A subsystem configured to upsample a video data stream encoded in YCrCb format 4:2:0 (also termed YUV 4:2:0) performs an algorithm upon a two-by-two group of subsampled pixels. The subsystem computes an inside probability that the chrominance of a target pixel is a close match to the chrominance inside the group of four pixels. The subsystem further computes three weighting factors relating the chrominance of the target pixel to each of three adjacent pixels in an upsampled four-by-four pixel group. The subsystem then computes an outside estimate of the chrominance based on the weighting factors. Finally, the subsystem computes the chrominance of the target pixel based on the inside probability, the outside estimate, and the subsampled chrominance. The subsystem performs the algorithm upon all two-by-two groups of four pixels within a subsampled YUV 4:2:0 video data stream and generates an upsampled YUV 4:4:4 video data stream.