There are provided a semiconductor memory device and a manufacturing method thereof. The semiconductor memory device includes a first select group and a second select group isolated from each other by an isolation insulating layer; an upper gate stack structure extending to overlap with the first select group, the isolation insulating layer, and the second select group; channel structures extending to penetrate the first select group, the second select group, and the upper gate stack structure; and a vertical connection structure spaced apart from the first select group, the second select group, and the upper gate stack structure, the vertical connection structure extending in parallel to the channel structures.