Emitting stripe pattern light, whose intensity changes periodically, onto a subject while shifting the phase of the pattern by π/2 from a reference phase of 0 until the phase is shifted one cycle to receive reflected light by an image sensor at each of phases 0, π/2, π, and 3π/2. From received light signals obtained at four phases, calculating a phase difference between the stripe pattern light and reflected light thereof with respect to each light receiving element to calculate distance information representing a distance to the subject based on the phase difference. In this case, the distance information is calculated only from a light receiving element having an absolute value of the difference between a first added signal of received light signals at 0 and π and a second added signal of received light signals at π/2 and 3π/2 smaller than a specific value.