The correlation of optical images with SEM images includes acquiring a full optical image of a sample by scanning the sample with an optical inspection sub-system, storing the full optical image, identifying a location of a feature-of-interest present in the full optical image with an additional sources, acquiring an SEM image of a portion of the sample that includes the feature at the identified location with a SEM tool, acquiring an optical image portion at the location identified by the additional source, the image portions including a reference structure, correlating the image portion and the SEM image based on the presence of the feature-of-interest and the reference structure in both the image portions and the SEM image, and transferring a location of the feature-of-interest in the SEM image into the coordinate system of the image portion of the full optical image to form a corrected optical image.