A head band structure of headset includes a retractable adjusting block and a head band group. The retractable adjusting block includes an upper cover, a sliding slice, an elastic piece and a lower cover. The sliding slice has at least one blocking portion. A substantial middle of the elastic piece is arched upward to form a convex surface. The head band group is connected with the retractable adjusting block. The head band group has a head band, and two end covers disposed to two opposite ends of the head band. A surface of each end cover has a waveform locking portion, and at least one stopping frame disposed at one end of the waveform locking portion. The convex surface keeps contacting with the waveform locking portion.