An augmented reality system comprises one or more sensors to capture a set of map points pertaining to the real world, wherein the set of map points are captured through a plurality of augmented reality systems, and a processor to determine a position of a plurality of keyframes that captured the set of map points, and to determine a set of new map points based at least in part on the captured set of map points and the determined position of the plurality of keyframes.