At least one dielectric pad layer is formed on a semiconductor-on-insulator (SOI) substrate. A deep trench is formed in the SOI substrate, and a combination of an outer electrode, a node dielectric, and an inner electrode are formed such that the top surface of the inner electrode is recessed below the top surface of a buried insulator layer of the SOI substrate. Selective epitaxy is performed to fill a cavity overlying the inner electrode with an epitaxial semiconductor material portion. A top semiconductor material layer and the epitaxial semiconductor material portion are patterned to form a fin structure including a portion of the top semiconductor material layer and a portion of the epitaxial semiconductor material portion. The epitaxial semiconductor material portion functions as a conductive strap structure between the inner electrode and a semiconductor device to be formed on the fin structure.