An inspection region 201 of a sample 2 is scanned with light, an optical image formed of the scanning light is acquired according to progression of scanning with the light, a reference image as a reference for the acquired optical image is created according to progression of acquisition of the optical image, the acquired optical image is compared to the reference image for the optical image to detect first defects in the pattern according to progression of the acquisition of the optical image, second defects caused by an erroneous operation of an inspection system 1 are detected based on a distribution of differences between the acquired optical image and the reference image during progression of detection of the first defects, and an inspection is stopped when the second defects are detected.