A method includes depositing a gate dielectric layer over a substrate. A gate electrode layer, a protection oxide layer, and a hard mask are sequentially deposited over the gate dielectric layer. The gate electrode layer and the protection oxide layer are patterned by using the hard mask as an etching mask to form a gate structure over the gate dielectric layer. An etching process is performed to remove the hard mask and thin the protection oxide layer after forming the gate structure.