Provided is a semiconductor device which performs a refresh operation by sequentially counting a refresh address including a main word line address, a mat address, and a sub word line address in order of the main word line address, the mat address, and the sub word line address. The semiconductor device includes a control signal generation unit configured to activate, latch, and output a toggle control signal when a delayed refresh signal is inputted at the initial stage, deactivate and output the toggle control signal after additionally counting a redundancy word line address when counting of the main word line address with respect to the mat address is completed, and then activate, latch, and output the toggle control signal when the delayed refresh signal is inputted.