A collision avoidance system for deciding whether an autonomous avoidance maneuver should be performed in order to avoid a mid-air collision between a host aerial vehicle equipped with the system and an intruding aerial vehicle. At least one electro-optical sensor captures consecutive images of an intruding vehicle such that the vehicle manifests itself as a target point in the images. An image processor estimates the azimuth angle, elevation angle and a first time-to-collision estimate of the time to collision between the host vehicle and the intruding vehicle. The first time-to-collision estimate is estimated based on scale change in the target point between at least two of said consecutive images. A tracking filter is arranged to estimate a second time-to-collision estimate using the azimuth angle, the elevation angle and the first time-to-collision estimate estimated by the image processor as input parameters. A collision avoidance module is arranged to decide whether or not an avoidance maneuver should be performed based on any of at least one parameter, of which at least one is indicative of said second time-to-collision estimate.