The present invention discloses an image sharpness processing apparatus and an image sharpness processing method thereof. The apparatus comprises an image capturing module and a processing module. The image capturing module captures an image having a plurality of pixels. The processing module gains a characteristic value corresponding to each pixel by analyzing each pixel in the image. The processing module calculates a first sharpening compensation value and a second sharpening compensation value of each pixel by using a first sharpening algorithm and a second sharpening algorithm respectively, and determines a weight value of the first sharpening algorithm and the second sharpening algorithm by the characteristic value. The processing module calculates a third sharpening compensation value according to the first sharpening compensation value and the second sharpening compensation value, so as to adjust the sharpness of the image.