A semiconductor memory device includes a memory cell array configured to include a plurality of word lines, a dock enable buffer configured to receive a clock enable signal, a plurality of command buffers configured to receive a plurality of commands, a refresh control unit configured to sequentially activate the plurality of word lines in a self-refresh mode, a command decoder configured to decode the clock enable signal and the plurality of commands, and to allow the refresh control unit to enter the self-refresh mode or exit from the self-refresh mode, and a buffer control unit configured to disable the plurality of command buffers when the clock enable signal is deactivated, and to enable the plurality of command buffers when the refresh control unit exits from the self-refresh mode.