Data associated with pixels obscured by annotations in an annotated image is stored using an annotation mask enabling annotation-aware applications to reconstruct the original image, while annotation-unaware applications can display the annotated image with optimized increase in extra data to be stored. The annotation mask, same size as the image, is initialized to a monochrome default and the color depth information for obscured pixels saved within the annotation mask. Because a majority area of the annotation mask is monochrome, it compresses well and the obscured image content can be saved in a side channel, metadata header, or a separate file. During reconstruction, changes to annotated image such as rotation, size change, and the like, may be detected and accounted for when obscured image content is carried over to the annotated image for reconstruction.