Disclosed in the application is a handheld three-dimensional ultrasound imaging method, comprising using a handheld ultrasound probe to scan a part to be tested; obtaining a three-dimensional position and angle information corresponding to ultrasound images according to a positioning reference device adapted to be arranged on the part to be tested; obtaining a moving distance and a rotation angle of the handheld ultrasound probe according to a material interference of the localization pattern with the ultrasonic signal; extracting information of the localization pattern from the ultrasound image as positioning information; restoring the ultrasound image to a state without interference from the localization pattern; performing 3D image reconstruction and display of the ultrasound image. The large spatial positioning system in an existing three-dimensional ultrasound imaging system is changed into a portable spatial positioning system that can be used at any time, so that handheld three-dimensional ultrasound imaging can be widely applied.