A face image processing apparatus includes: a lighting portion including a first polarizer which polarizes light in a first direction and a light emitter which emits, through the first polarizer, infrared light; an image capturing portion including a second polarizer which polarizes light in a second direction perpendicular to the first direction and an image capturing unit which captures images through the second polarizer; and an image processing portion which detects candidates of eyes using a first image captured when the lighting portion emits the infrared light and a second image captured when the lighting portion does not emit the infrared light. The image processing portion determines, as an eye, a candidate having a hyperbolic or cross shaped pattern present in the first image but not present in the second image.