A method for modulating a work function of a semiconductor device having a metal gate structure including the following steps is provided. A first stacked gate structure and a second stacked gate structure having an identical structure are provided on a substrate. The first stacked gate structure and the second stacked gate structure respectively include a first work function metal layer of a first type. A patterned hard mask layer is formed. The patterned hard mask layer exposes the first work function metal layer of the first stacked gate structure and covers the first work function metal layer of the second stacked gate structure. A first gas treatment is performed to the first work function metal layer of the first stacked gate structure exposed by the patterned hard mask layer. A gas used in the first gas treatment includes nitrogen-containing gas or oxygen-containing gas.