An apparatus for processing an encoded audio signal, which includes a sequence of access units, each access unit including a core signal with a first spectral width and parameters describing a spectrum above the first spectral width, has a demultiplexer generating, from an access unit of the encoded audio signal, the core signal and a set of the parameters, an upsampler upsampling the core signal of the access unit and outputting a first upsampled spectrum and a timely consecutive second upsampled spectrum, the first upsampled spectrum and the second upsampled spectrum, both, having a same content as the core signal and having a second spectral width being greater than the first spectral width of the core spectrum, a parameter converter converting parameters of the set of parameters of the access unit to obtain converted parameters, and a spectral gap filling processor processing the first upsampled spectrum and the second upsampled spectrum using the converted parameters.