In one embodiment, a method determines an encoded plurality of layers for a scalable video stream for a video where each of the plurality of layers includes a respective layer parameter setting. An additional layer for the plurality of layers is determined where the additional layer enhances a base layer in the plurality of layers and the respective layer parameter settings for the encoded plurality of layers do not take into account the additional layer. The method then determines an additional layer parameter setting for the additional layer. The additional layer parameter setting specifies a relationship between the additional layer and at least a portion of the plurality of layers where the additional layer parameter setting is used to decode the additional layer and the at least a portion of the plurality of layers.