A system for adapting virtual reality (VR) or augmented reality (AR) content for learning based on a user's interests includes a personalization engine that determines a user's interests and knowledge level regarding a topic, a content modifier that modifies VR or AR content related to the topic according to the users interests and knowledge level, an object selector that selects VR and AR models used to teach the topic during a VR or AR session and modifies the VR and AR models with the modified content based on the user's interests and knowledge level; a VR engine that renders the modified VR and AR models into VR or AR images, and a VR display that displays the VR or AR images. The user's engagement is measured during interaction with the VR display and used to refine and improve an understanding of the user's interest and knowledge level regarding the topic.