Systems and methods of providing stereo depth cameras for head-mounted display systems that require less memory and/or processing power. The stereo depth camera may include a left camera and a right camera spaced apart from each other by a distance. Each of the left and right cameras may be skewed outward by a non-zero angle from a forward direction of the head-mounted display system to provide a relatively wide field of view for the stereo depth camera. Each of the left and right cameras may include a camera sensor array and a camera lens positioned forward of the camera sensor array. Each of the camera lenses may include an optical axis that is laterally offset from the center of the associated camera sensor array toward a center of the support structure to center the left camera lens substantially on a scene center or principal point.