Motion compensated temporal prediction is well-known for redundancy reduction in video signal compression. A prediction error signal and a motion vector for a current block are transmitted, describing the location of the predictor block in the reference picture. A problem solved by the invention is to encode and decode video signals using less or no mo- tion vector data side information. This is achieved through a temporal prediction which exploits common knowledge available in both encoder and decoder, requiring that the decoder performs a motion estimation, too. A common baseline motion estimation process is used at both encoder and decoder on the same set of video data that have already been compressed before and, hence, are available at both encoder and decoder, leading to identical motion parameters being generated in the decoder. These common motion parameters are used for generating a default temporal prediction without the need to transmit related motion vectors from the encoder to the decoder.