A method and system of vehicle localization. Sensed data representing an environment of the vehicle is received from at least one sensor. A top view image of the environment is generated from the sensed data representing the environment of the vehicle. A vehicle localization is determined using a top view from a plurality of previously captured top view images that matches the generated top view and the sensed data. An output is generated including the vehicle localization.