In inter prediction that is performed by partitioning a first block, which is obtained by partitioning each picture, into one or more second blocks, a spatial merge candidate generating unit derives a spatial merge candidate without referring to a block included in a first block that includes a second block. In case of a mode where a coding block is divided by a horizontal border into prediction blocks arranged vertically, the reference index derivation unit of a time merge candidate sets reference index information of a time merge candidate to a value of reference index information of an encoded prediction block adjacent to a left edge of a prediction block subject to encoding.