A speech data retrieval apparatus (10) includes a speech database (1), a speech recognition unit (2), a confusion network creation unit (3), an inverted index table creation unit (4), a query input unit (6), a query conversion unit (7) and a label string check unit (8). The speech recognition unit (2) reads speech data from the speech database (1), carries out a speech recognition process with respect to the read speech data, and outputs a result of speech recognition process as a lattice in which a phoneme, a syllable, or a word is a base unit. The confusion network creation unit (3) creates a confusion network based on the output lattice and outputs the result of speech recognition process as the confusion network. The inverted index table creation unit (4) creates an inverted index table based on the output confusion network. The query input unit (6) receives a query input by a user, carries out a speech recognition process with respect to the received query, and outputs a result of speech recognition process as a character string. The query conversion unit (7) converts the output character string into a label string in which a phoneme, a syllable, or a word is a base unit. The label string check unit (8) checks the label string against the inverted index table and retrieves speech data which is included in both of the label string and the speech database (1).