A focus detection apparatus includes an image pickup element 106 that takes an object image, an object recognition circuit 109 that compares an obtained image with a previously stored image to detect a position of a main object, an AF sensor 101 that detects an in-focus state of a plurality of positions, a reliability determination circuit 110 that determines a reliability of a detection result of the in-focus state, a determination part 100 that determines a position where the reliability of the detection result of the in-focus state is the highest based on a determination result of the reliability determination circuit, and a selector 114 that selects one of the detection results of the in-focus state of the position of the main object and of the in-focus state of the position where the reliability of the detection result of the in-focus state is highest to focus the photographic optical system.