A system, computer-readable medium, and method for localization and mapping for an autonomous vehicle are provided. The system may obtain an image. The system may assign labels to one or more objects of the image. The system may also obtain a point cloud. The system may determine one or more object clusters of the point cloud and associate the labels assigned to the one or more objects of the image with points of the object clusters of the point cloud. The system may further identify three-dimensional (3D) objects of the point cloud based on the labels associated with the points of the object clusters. In some aspects, the system may remove dynamic traffic participants from the point cloud based on the identified 3D objects and/or perform a simultaneous localization and mapping operation on the point cloud after removing the dynamic traffic participants.