A method of manufacturing a nonvolatile semiconductor memory device includes the steps of preparing a wafer having multiple memory cells, each memory cell having a gate electrode formed on a semiconductor substrate, charge storage units formed on both sides of the gate electrode, lightly doped regions formed beneath the charge storage units, respectively, in the upper part of the semiconductor substrate, and highly doped regions formed in a pair of regions sandwiching a region underneath the gate electrode and the lightly doped regions in between; erasing data stored in the charge storage units electrically; and treating the wafer at a high temperature for a predetermined period of time.