A method for fabricating semiconductor device includes the steps of: providing a substrate having a first region, a second region, and a third region; forming a first gate structure on the first region, a second gate structure on the second region, and a third gate structure on the third region; forming an interlayer dielectric (ILD) layer around the first gate structure, the second gate structure, and the third gate structure; removing the first gate structure, the second gate structure, and the third gate structure to form a first recess, a second recess, and a third recess; forming a first interfacial layer in the first recess, the second recess, and the third recess; removing the first interfacial layer in the second recess; and forming a second interfacial layer in the second recess.