A vehicular trailer assist system includes a camera disposed at a rear portion of a vehicle, a plurality of sensors, and an electronic control unit (ECU) having a processor for processing image data captured by the camera and sensor data captured by the plurality of sensors. The ECU, responsive to processing of image data captured by the camera, determines a trailer angle of the trailer relative to the vehicle. The ECU, responsive to processing of sensor data captured by the plurality of sensors, predicts a yaw rate of the vehicle and a trailer angle of the trailer relative to the vehicle at a future time. The ECU, responsive to predicting the yaw rate, determines a corrected vehicle yaw rate based on processing of captured image data and captured sensor data, and determines a corrected trailer angle, and determines a vehicle heading and position and a trailer heading and position.