A conductive strap structure in lateral contact with a top semiconductor layer is formed on an inner electrode of a deep trench capacitor. A cavity overlying the conductive strap structure is filled a dielectric material to form a dielectric capacitor cap having a top surface that is coplanar with a topmost surface of an upper pad layer. A semiconductor mandrel in lateral contact with the dielectric capacitor cap is formed. The combination of the dielectric capacitor cap and the semiconductor mandrel is employed as a protruding structure around which a fin-defining spacer is formed. The semiconductor mandrel is removed, and the fin-defining spacer is employed as an etch mask in an etch process that etches a lower pad layer and the top semiconductor layer to form a semiconductor fin that laterally wraps around the conductive strap structure. An access finFET is formed employing two parallel portions of the semiconductor fin.