According to one embodiment, in a semiconductor memory device, a source region and a drain region are disposed away from each other in the semiconductor layer. A tunnel insulating film is formed between the source region and the drain region on the semiconductor layer. A charge accumulating film includes an oxide cluster and is formed on the tunnel insulating film. A block insulating film is formed on the charge accumulating film. A gate electrode is formed on the block insulating film. The oxide cluster includes either Zr or Hf, and further contains at least one element selected from Ti, V, Cr, Mn, Fe, Co, Ni, Cu, Zn, Nb, Mo, Tc, Ru, Rh, Pd, Ag, Cd, Ta, W, Re, Os, Ir, Pt, Au and Hg.