Disclosed are an intra prediction mode based image processing method, and an apparatus therefor. In particular, a method for processing an image on the basis of an intra prediction mode comprises the steps of: inputting, in most probable mode (MPM) candidates, intra prediction modes of a plurality of neighboring blocks which neighbor a current block, and then deriving the MPM candidates by inputting, in the MPM candidates, intra prediction modes within substitute modes consisting of a plurality of intra prediction modes, to which priorities are given, according to the priorities until the total number of MPM candidates is satisfied; deriving, from the MPM candidates, an intra prediction mode applied to the current block using an MPM index indicating a specific intra prediction mode; and generating a prediction block of the current block by applying the derived intra prediction mode.