Invention Grant
- Patent Title: Method, electronic device and computer readable medium for information processing for accelerating neural network training
-
Application No.: US16660259Application Date: 2019-10-22
-
Publication No.: US11640528B2Publication Date: 2023-05-02
- Inventor: Zhiyu Cheng , Baopu Li , Yingze Bao
- Applicant: Baidu USA LLC
- Applicant Address: US CA Sunnyvale
- Assignee: Baidu USA LLC
- Current Assignee: Baidu USA LLC
- Current Assignee Address: US CA Sunnyvale
- Agency: Nixon Peabody LLP
- Main IPC: G06N3/08
- IPC: G06N3/08 ; G06N20/00 ; G06N3/048

Abstract:
A method for information processing for accelerating neural network training. The method includes: acquiring a neural network corresponding to a deep learning task; and performing iterations of iterative training on the neural network based on a training data set. The training data set includes task data corresponding to the deep learning task. The iterative training includes: processing the task data in the training data set using a current neural network, and determining, based on a processing result of the neural network on the task data in a current iterative training, prediction loss of the current iterative training; determining a learning rate and a momentum in the current iterative training; and updating weight parameters of the current neural network by gradient descent based on a preset weight decay, and the learning rate, the momentum, and the prediction loss in the current iterative training. This method achieves efficient and low-cost deep learning-based neural network training.
Public/Granted literature
Information query