A wearable simulation system worn by a user outputs optical and acoustic signals that cause the user to perceive a simulation. The simulation may be an immersive virtual world or, alternatively, an augmentation to the real world. The wearable simulation system also captures acoustic and optical signals from the environment around the user and then selectively incorporates these signals into the simulation. Accordingly, the user may perceive the environment to a limited extent, while still remaining engaged with the simulation. At least one advantage of the disclosed techniques is that the wearable simulation system does not completely interfere with user perception of the environment, thereby reducing the risk of the user missing important real-world events or being otherwise unaware of such real-world events.