The image processing device performs noise removal processing for an input image. The image processing device comprises a filter setting unit configured to set first and second filter factors by using a first target pixel value of a noise processing target pixel to be targeted to the processing and a reference pixel value of a plurality of reference pixels surrounding the noise processing target pixel and a noise removal unit configured to calculate a corrected pixel value by sequentially adding a luminance component of the reference pixel value weighted with the first filter factor and sequentially adding a chroma component of the reference pixel value weighted with the second filter factor, and remove noise of the noise processing target pixel by using the corrected pixel value. The input image has pixel values according to Bayer pattern format.