In one embodiment, a system of an ADV perceives an object in a perspective view of the ADV using a plurality of sensors mounted on the ADV including capturing an image of the object. The system determines a two-dimensional (2D) bounding box for the object and a perceived distance from the ADV to the object. The system searches, within an image space of the image, a dynamic pitch angle based on the 2D bounding box and the perceived distance from the ADV to the object by modifying at least a previous extrinsic calibration value to calculate a distance. The system determines a distance from the ADV to a point on a ground plane of the perspective view of the ADV based on the determined dynamic pitch angle. The system generates a driving trajectory based on the determined distance for the point on the ground plane to control the ADV.