The shadows and highlights regions of an image can be automatically optimized in a localized manner. A mask can be generated that can automatically identify local regions of a digital image as highlight regions or shadow regions. Different processing can then be automatically applied to highlight regions separately from the shadow regions. Luminance histograms can be obtained for the overall digital image, as well as those portions of the digital image that are in the highlights regions and, separately, the shadows regions. The moments of those histograms, including the average and the variance, can be compared to target moments and processing can continue to be applied to highlights regions and, separately, shadows regions, until one or more target moments are achieved. Target moments can be generated from original moments of the original image histograms based on relationships generated from a prior manual optimization of a reference set of images.