A focus detection apparatus comprises: a photoelectric converter including a pair of line sensors to output electrical signals; a selection unit constructed to select one of focus detection points; a control unit constructed to divide the pair of line sensors into a plurality of pairs of blocks in accordance with different divisional patterns which make the resultant plurality of pairs of blocks respectively corresponding to the focus detection points, and read out electrical signals from each pair of blocks when the levels of the electrical signals of the pair of blocks reach a level suitable for focus detection; frame memories, constructed to store the read out electrical signals, corresponding to the divisional patterns; and a focus detection unit constructed to detect a focus state of an imaging lens, on the basis of a phase difference between electrical signals of the pair of blocks corresponding to the selected focus detection point.