According to one embodiment, a semiconductor memory device comprises a cell transistor includes a first gate electrode buried in a semiconductor substrate and a first diffusion layer and a second diffusion layer formed to sandwich the first gate electrode, a first lower electrode formed on the first diffusion layer, a magnetoresistive element formed on the first lower electrode to store data according to a change in a magnetization state and connected to a bit line located above, a second lower electrode formed on the second diffusion layer, and a first contact formed on the second lower electrode and connected to a source line located above. A contact area between the second lower electrode and the second diffusion layer is larger than a contact area between the first contact and the second lower electrode.