The present disclosure relates to applying techniques similar to those used in neural network language modeling systems to a content recommendation system. For example, by associating consumed media content to words of a language model, the system may provide content predictions based on an ordering. Thus, the systems and techniques described herein may produce enhanced prediction results for recommending content (e.g. word) in a given sequence of consumed content. In addition, the system may account for additional user actions by representing particular actions as punctuation in the language model.