A method includes forming an interposer, which includes a semiconductor substrate, and an interconnect structure over the semiconductor substrate. The method further includes bonding a device die to the interposer, so that a first metal pad in the interposer is bonded to a second metal pad in the device die, and a first surface dielectric layer in the interposer is bonded to a second surface dielectric layer in the device die. The method further includes encapsulating the device die in an encapsulating material, forming conductive features over and electrically coupling to the device die, and removing the semiconductor substrate. A part of the interposer, the device die, and portions of the conductive features in combination form a package.