An approach is provided in which a knowledge manager detects an annotation error while attempting to annotate a document and, as such, segments the document into multiple document segments. The knowledge manager then detects another annotation error while attempting to annotate the multiple document segments, resulting in a set of unannotated document segments. In turn, the knowledge manager generates a notification that identifies the set of unannotated document segments.