This disclosure is directed to calibrating sensors mounted on an autonomous vehicle. First image data and second image data representing an environment can be captured by first and second cameras, respectively (and or a single camera at different points in time). Point pairs comprising a first point in the first image data and a second point in the second image data can be determined and projection errors associated with the points can be determined. A subset of point pairs can be determined, e.g., by excluding point pairs with the highest projection error. Calibration data associated with the subset of points can be determined and used to calibrate the cameras without the need for calibration infrastructure.