A method and apparatus for predicting reference data transfer scheme for motion estimation. The method includes computing, via the processor, hypothetical rectangle region in reference frame containing all the predicting and reference data for doing motion estimation search around the region, if the macroblock is not the first in a row, utilizing overlap with previously fetched reference data, computing overlap with previously fetched reference data, and transferring needed data, invalidating any predictor, wherein the predictor is not part of the fetched data, and regulating the motion estimation and setting the motion vector to an effective value based on the fetched and computed data.