A semiconductor memory device includes a plurality of first conductor layers that are stacked in a first direction; a first pillar including a first semiconductor layer and extending through the first conductor layers in the first direction; a first charge storage layer that is provided between the first conductor layers and the first semiconductor layer; a plurality of second conductor layers that are stacked in the first direction above an uppermost conductor layer of the first conductor layers; a second pillar including a second semiconductor layer and extending through the second conductor layers in the first direction, the second semiconductor layer electrically connected to the first semiconductor layer; and a conductor pillar or film extending in the first direction through the second conductor layers other than a lowermost layer of the second conductor layers and being in contact with a respective upper surface of each of the second conductor layers.