A method performed by a video encoder for encoding a current picture belonging to a temporal level identified by a temporal_id. The method includes determining a Reference Picture Set (RPS) for the current picture indicating reference pictures that are kept in a decoded picture buffer (DPB) when decoding the current picture, and when the current picture is a temporal switching point. The method further comprises operating to ensure that the RPS of the current picture includes no picture having a temporal_id greater than or equal to the temporal_id of the current picture.