A method of generating reference spectra includes polishing a first substrate in a polishing apparatus, measuring a sequence of spectra from the first substrate during polishing with an in-situ optical monitoring system, for each spectrum in the sequence of spectra, determining a best matching reference spectrum from a first plurality of first reference spectra to generate a sequence of reference spectra, calculating a value of a metric of fit of the sequence of spectra to the sequence of reference spectra, comparing the value of the metric of fit to a threshold value and determining whether to generate a second library based on the comparison, and if the second library is determined to be generated, storing the sequence of spectra as a second plurality of reference spectra.