A three-dimensional object detection device includes an image capturing unit, a detection area setting unit, an image conversion unit, a three-dimensional object detection unit, a reference image area setting unit, a contour information extraction unit, a reference sharpness calculation unit and a subject sharpness calculation unit. The image conversion unit converts a viewpoint of an image obtained by the image capturing unit to create a bird's-eye view image. The reference image area unit sets a reference image area for extracting contour information by the contour information extraction unit. The reference sharpness calculation unit calculates a reference sharpness using contour information of the reference image area. The subject sharpness calculation unit calculates a subject sharpness using contour information of the detection area. The three-dimensional object detection unit calculates a final sharpness based on the reference sharpness and the subject sharpness, and sets a difference threshold value based on the final sharpness.