A speaker identification device includes: a primary speaker identification unit that computes, for each pre-stored registered speaker, a score that indicates the similarity between input speech and speech of the registered speakers; a similar speaker selection unit that selects a plurality of the registered speakers as similar speakers according to the height of the scores thereof; a learning unit that creates a classifier for each similar speaker by sorting the speech of a certain similar speaker among the similar speakers as a positive instance and the speech of the other similar speakers as negative instances; and a secondary speaker identification unit that computes, for each classifier, a score of the classifier with respect to the input speech, and outputs an identification result.