By eliminating the necessity of a prior step for cleaning a sintered magnet before adhering Dy and/or Tb to the surface of the sintered magnet S, the productivity of a permanent magnet having diffused Dy and/or Tb into grain boundary phase is improved. Iron-boron-rare earth based sintered magnet (S) disposed in a processing chamber (20) is heated to a predetermined temperature. An evaporating material (V) which is made of a hydride containing at least one of Dy and Tb is disposed in the same or in another processing chamber and is evaporated to cause the evaporated evaporating material to the surface of the sintered magnet. Metal atoms of Dy and/or Tb are diffused into grain boundary phase of the sintered magnet.