A semiconductor device includes conductive pillars disposed vertically over a seed layer, a conformal insulating layer formed over the conductive pillars, and a conformal conductive layer formed over the conformal insulating layer. A first conductive pillar, the conformal insulating layer, and the conformal conductive layer constitute a vertically oriented integrated capacitor. The semiconductor device further includes a semiconductor die or component mounted over the seed layer, an encapsulant deposited over the semiconductor die or component and around the conformal conductive layer, and a first interconnect structure formed over a first side of the encapsulant. The first interconnect structure is electrically connected to a second conductive pillar, and includes an integrated passive device. The semiconductor device further includes a second interconnect structure formed over a second side of the encapsulant opposite the first side of the encapsulant.