Probabilistic parsing is described for calculating information about the structure of text and other ordered sequences of items to enable downstream systems such as machine translation systems, information retrieval systems, document classification systems and others to use the structure information. In various embodiments, a parsing inference component comprises inference algorithm(s) compiled from a probabilistic program which defines a stochastic process for generating text or other ordered sequences of items. In examples, the parsing inference component receives one or more observations or examples of text that are compatible with the stochastic process defined by the probabilistic program. The parsing inference component may apply the inference algorithms to the text to update one or more probability distributions over strings or other values relevant to the parse. In some examples, the parsing inference component uses the inference results to complete partial examples to assist a user with information retrieval tasks.