Embodiments of 3D memory structures and methods for forming the same are disclosed. The fabrication method includes disposing an alternating dielectric stack on a substrate, wherein the alternating dielectric stack having first and second dielectric layers alternatingly stacked on top of each other. Next, a plurality of contact openings can be formed in the alternating dielectric stack such that a dielectric layer pair can be exposed inside at least one of the plurality of contact openings. The method further includes forming a film stack of alternating conductive and dielectric layers by replacing the second dielectric layer with a conductive layer, and forming a contact structure to contact the conductive layer in the film stack of alternating conductive and dielectric layers.