Semiconductor devices are provided, and includes a substrate, a first gate structure and a second gate structure over the substrate, a first hard mask on the first gate structure and a second hard mask on the second gate structure and a third hard mask. The third hard mask is disposed in a dielectric layer between the first gate structure and the second gate structure and disposed between the first hard mask and the second hard mask.