A system and method for generating a three-dimensional image using depth layers is provided. A plurality of depth layers may be generated from the image, where each depth layer has a height, a width, and a thickness. Certain regions of the image may be assigned to one of the plurality of depth layers. A depth map may be generated based on the depth layers. Further, a disparity map may be generated based on both the depth map and the depth layers. A stereo view of the image may then be rendered based on the disparity map.