A method and apparatus for rendering the lighting of virtual objects in an augmented reality display. The method includes determining local and ambient light sources based on data provided by one or more light sensors. The light in the physical lighting environment is accounted for by attributing the light to local light sources and/or ambient light sources. A synthesized physical lighting environment is constructed based on the light characteristics of the local and/or ambient light sources, and is used in properly rendering virtual objects in the augmented reality display.