A method of manufacturing a semiconductor device includes forming a gate electrode, a source region and a drain region, forming a first metal layer, forming silicide layers by first annealing, removing a remainder of the first metal layer after the first annealing, performing a second annealing, forming a second metal layer, performing a third annealing, and removing a remainder of the second metal layer.