Results of automatic detection of dirt or other non-steady defects in a sequence of digitized image frames can be unreliable. Here, a determination of a detection of a defective object to be replaced by a replacement pattern in a frame of a sequence of image frames as a misdetection is presented that comprises determining a value of a first similarity measure between a boundary of the detected defective object and a boundary of the replacement pattern and determining a value of a second similarity measure between the detected defective object and the replacement pattern. The detection of the defective object is determined as a misdetection if at least one of the following holds true: the value of the first similarity measure is outside of a corresponding first similarity acceptance range and the value of the second similarity measure is inside of a corresponding second similarity acceptance range.