The capability to render and navigate three-dimensional panoramic images in a virtual three-dimensional environment so as to create an immersive three-dimensional experience is provided. Such a capability can present a three-dimensional photographic experience of the real world that is seamlessly integrated with the virtual three-dimensional environment. Depth values associated with the panoramic images may be used to create three-dimensional geometry, which can be rendered as part of the virtual three-dimensional environment. Further, such a capability can enable a user to roam freely through the environment while providing a more natural free-form exploration of the environment than existing systems.