A semiconductor memory device includes a substrate; a stacked body on the substrate and including a first stacked body formed of stacked first electrode layers and a second stacked body on the first stacked body and including a second electrode layer; a hole passing through the stacked bodies in a first direction and having a first insulator, and a channel film between the first insulator and first electrode layers and between the first insulator and second electrode layer and having first and second portions facing each other, with the first insulator placed therebetween. A first memory between the first electrode layers and the first portion and a second memory between the first electrode layers and the second portion are insulated. A third memory between the second electrode layer and the first portion and a fourth memory between the second electrode layer and the second portion are connected.