A method and apparatus for neural network based cross component prediction with low-bit precision during encoding or decoding of an image frame or a video sequence, which may include reconstructing a chroma component based on a received luma component using a pre-trained deep neural network (DNN) cross component prediction (CCP) model for chroma prediction, and updating a set of parameters of the pre-trained DNN CCP model with low-bit precision. The method may also include generating an updated DNN CCP model for chroma prediction with low-bit precision based on at least one video sequence, and using the updated DNN CCP model for cross component prediction of the at least one video sequence at reduced processing time.