A custom gesture collection and recognition system having a machine learning accelerator includes a transmission unit, a first reception chain, a second reception chain, a customized gesture collection engine and a machine learning accelerator. The transmission unit transmits a transmission signal to detect a gesture. The first reception chain receives a first signal and generates first feature map data corresponding to the first signal. The second reception chain receives a second signal and generates second feature map data corresponding to the second signal. The first signal and the second signal are generated by the gesture reflecting the transmission signal. The customized gesture collection engine generates gesture data according to at least the first feature map data and the second feature map data. The machine learning accelerator performs machine learning with the gesture data. The accuracy and correctness of gesture recognition may be improved by means of machine learning.