A nonvolatile semiconductor memory device includes a source region and a drain region provided apart from each other in a semiconductor substrate, a first insulating film provided on a channel region between the source region and the drain region, a charge storage layer provided on the first insulating film, a second insulating film provided on the charge storage layer and including a stacked structure of a lanthanum aluminum silicate film and a dielectric film made of silicon oxide or silicon oxynitride, and a control gate electrode provided on the second insulating film.