According to one embodiment, a magnetic memory device includes a first insulating film provided on a semiconductor region, and having a portion located in a memory cell array area and thicker than a portion located in a peripheral circuit area, a plurality of conductive plugs located in the memory cell array area and provided in the first insulating film, stacked structures located in the memory cell array area, provided on the conductive plugs, and each having layers including a magnetic layer, and transistors located in the peripheral circuit area, and each including a gate electrode provided on the semiconductor region and covered with the first insulating film, wherein a thickness t0 from a main surface of the semiconductor region to a lower surface of each stacked structure is greater than a predetermined value.