An image encoding apparatus includes a mode determining unit and an encoding unit. The mode determining unit determines a first operating mode and a second operating mode of an adaptive filter which is applied to a block in an input image in decoding. In this regard, the mode determining unit determines the first operating mode based on a first signal of the input image, and determines the second operating mode based on a second signal of the input image. The encoding unit encodes the second operating mode by assigning a code with a lower bit number to the second operating mode when the second operating mode has higher similarities to the first operating mode.