In one embodiment, a method of manufacturing a semiconductor memory device is disclosed. The method can comprise forming a tunnel insulating film on a substrate, forming a charge storage layer including a conductor on the tunnel insulating film, forming an isolation trench which isolate the charge storage layer and the tunnel insulating film in the substrate, embedding an isolation insulating film in the isolation trench, removing a native oxide film which is formed on a surface of the charge storage layer, and forming an insulating film on a surface of the isolation insulating film and the surface of the charge storage layer. The process from the removing the native oxide film to the forming the insulating film carried out in a manufacture apparatus in which an oxygen concentration is controlled.