An example device for filtering a decoded block of video data includes one or more processing units configured to construct a plurality of filters for classes of blocks of a current picture of video data. To construct the plurality of filters for each of the classes, the processing units are configured to determine a value of a flag that indicates whether a fixed filter is used to predict a set of filter coefficients of the class, and in response to the fixed filter being used to predict the set of filter coefficients, determine an index value into a set of fixed filters and predict the set of filter coefficients of the class using a fixed filter of the set of fixed filters identified by the index value.