A method and system for scaling an original image into a scaled image is disclosed. Rather than using the corresponding position in the original image to generate the pixels values of a current pixel in the scaled image, image scalers in accordance with the present invention, calculate a high frequency adjusted position based on the high frequency components of pixels near the corresponding position. Pixel values based on the high frequency adjusted position provide better picture quality for the scaled image than pixel values based on the corresponding position. Furthermore, some embodiments of the present invention also use sharpness compensation to further improve the picture quality of the scaled image.