The present invention relates to a simplified pipeline for Sample Adaptive Offset (SAO) and Adaptive Loop Filtering (ALF) in the in-loop decoding of a video encoder and a video decoder. According to the present invention, filter parameter setting regions and filtering processing windows are aligned, to reduce the required amount of memory for parameter sets necessary for delayed filtering. This is preferably achieved by a displacement of the filter parameter setting regions with respect to LCU boundaries in at least one (preferably: vertical) or both vertical and horizontal directions.