This specification describes techniques for verifying authenticity of an image. One example method includes identifying a baseline image depicting a baseline background; identifying a comparison image depicting a card; identifying a comparison background in the comparison image, wherein the comparison background is an area of the comparison image other than an area occupied by the card; determining a probability that the baseline background matches the comparison background; determining that the probability satisfies a verification threshold; and in response to determining that the probability satisfies a verification threshold, determining that the comparison image was acquired by capturing an image of a physical card corresponding to the card depicted in the comparison image.