A focus detection apparatus includes a first setting unit configured to set a first focus detection area and a second focus detection area, a first focus detection unit configured to perform focus detection of a phase difference detection method on each of the first and second focus detection areas by using the pair of parallax image signals, a reliability acquisition unit configured to obtain reliability of the focus detection of the first and second focus detection areas by the first focus detection unit, a second setting unit configured to determine a third focus detection area by using the reliability of the focus detection of the first and second focus detection areas, obtained by the reliability acquisition unit, and a second focus detection unit configured to perform focus detection by using the third focus detection area.