A method of controlling a row hammer swaps a first address entry with a second address entry having the smallest second access number and randomly swaps the first address entry with a third address entry having a third access number which is not the greatest value, in an address table representing a correlation between an address entry accessed during a row hammer monitoring time frame and an access number, thereby preventing a hacker-pattern row hammer aggression from being easily performed.