Certain embodiments involve selecting metrics that are representative of large metrics datasets and that are usable for efficiently performing anomaly detection. For example, metrics datasets are grouped into clusters based on, for each of the clusters, a similarity of data values in a respective pair of datasets from the metrics datasets. Principal component datasets are determined for the clusters. A principal component dataset for a cluster includes a linear combination of a subset of metrics datasets included in the cluster. Each representative metric is selected based on the metrics dataset having a highest contribution to a principal component dataset in the cluster into which the metrics dataset is grouped. An anomaly detection is executed in a manner that is restricted to a subset of the metrics datasets corresponding to the representative metrics.