A method for detecting and growing isolated holes in a document image having a plurality of pixels is provided. The method includes isolating the pixels of the image to form a plurality of windows, each window having a target pixel; identifying a hole growth factor to grow an isolated hole in the received image; using the hole growth factor to identify tiered pixel patterns from a plurality of predefined, tiered pixel patterns, wherein each of the tiered pixel patterns having a predetermined hole growth factor; comparing the pixels within each window to the pixel patterns within the identified tier to identify a match between the pixels within the window and at least one of the pixel patterns; and changing a pixel value of the target pixel, when a match is identified, to grow the isolated hole by the hole growth factor.