A lithographic apparatus includes a phase adjuster to adjust a phase of an optical wave traversing an optical element of the phase adjuster during exposure of a pattern on a substrate. In an embodiment, the optical element is a heat controllable optical element in a projection system of the lithographic apparatus. In use, the pattern is illuminated with an illumination mode including an off-axis radiation beam. This beam is diffracted into a number of first-order diffracted beams, one associated with a first pitch in the pattern, along a first direction, another associated with a second pitch along a different, second direction in the pattern. An area is identified where the first-order diffracted beam associated with the first pitch traverses the optical element. An image characteristic of an image of the pattern is optimized by calculating a desired optical phase of this first-order diffracted beam in relation to the optical phase of the other first-order diffracted beam. The phase adjuster is controlled to apply the desired optical phase to the first order diffracted beam.