Methods, systems, and computer program products for generating probabilistic annotations for entities and relations using reasoning and corpus-level evidence are provided herein. A method includes determining candidate entities and candidate relationships between entities from text contained within textual documents associated with a given domain; generating questions to support or refute legitimacy of a given one of the candidate entities and/or a given one of the candidate relationships; automatically determining an answer to each of the questions based on analysis of one or more additional items of knowledge pertaining to the given domain; computing a confidence score for each answer; and assigning a confidence value in support of the legitimacy of the given one of the candidate entities and/or the given one of the candidate relationships, wherein said confidence value is based on an aggregation of the confidence scores for the answers to the one or more questions.